Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 2.7.4, 3.0.0-alpha1
    • Component/s: hdfs
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      String/byte conversions may take either a Charset instance or its canonical name. One might think a Charset instance would be faster due to avoiding a lookup and instantiation of a Charset, but it's not. The canonical string name variants will cache the string encoder/decoder (obtained from a Charset) resulting in better performance.

      LOG4J2-935 describes a real-world performance boost. I micro-benched a marginal runtime improvement on jdk 7/8. However for a 16 byte path, using the canonical name generated 50% less garbage. For a 64 byte path, 25% of the garbage. Given the sheer number of times that paths are (re)parsed, the cost adds up quickly.

      1. HDFS-10662.patch
        10 kB
        Daryn Sharp
      2. HDFS-10662.patch.1
        9 kB
        Daryn Sharp
      3. HDFS-10662-branch-2.7.00.patch
        8 kB
        Zhe Zhang

        Issue Links

          Activity

          Hide
          daryn Daryn Sharp added a comment -

          trying to kick the pre-commit

          Show
          daryn Daryn Sharp added a comment - trying to kick the pre-commit
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          0 mvndep 0m 8s Maven dependency ordering for branch
          +1 mvninstall 6m 56s trunk passed
          +1 compile 1m 26s trunk passed
          +1 checkstyle 0m 32s trunk passed
          +1 mvnsite 1m 27s trunk passed
          +1 mvneclipse 0m 24s trunk passed
          +1 findbugs 3m 10s trunk passed
          +1 javadoc 1m 15s trunk passed
          0 mvndep 0m 7s Maven dependency ordering for patch
          +1 mvninstall 1m 20s the patch passed
          +1 compile 1m 25s the patch passed
          +1 javac 1m 25s the patch passed
          -0 checkstyle 0m 28s hadoop-hdfs-project: The patch generated 1 new + 127 unchanged - 0 fixed = 128 total (was 127)
          +1 mvnsite 1m 22s the patch passed
          +1 mvneclipse 0m 19s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 3m 27s the patch passed
          +1 javadoc 1m 12s the patch passed
          +1 unit 0m 53s hadoop-hdfs-client in the patch passed.
          -1 unit 58m 44s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 19s The patch does not generate ASF License warnings.
          86m 33s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics
            hadoop.cli.TestHDFSCLI
            hadoop.hdfs.server.datanode.TestDataNodeLifeline



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819178/HDFS-10662.patch
          JIRA Issue HDFS-10662
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux d785967c9ca1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / d84ab8a
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16198/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16198/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16198/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16198/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 8s Maven dependency ordering for branch +1 mvninstall 6m 56s trunk passed +1 compile 1m 26s trunk passed +1 checkstyle 0m 32s trunk passed +1 mvnsite 1m 27s trunk passed +1 mvneclipse 0m 24s trunk passed +1 findbugs 3m 10s trunk passed +1 javadoc 1m 15s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 1m 20s the patch passed +1 compile 1m 25s the patch passed +1 javac 1m 25s the patch passed -0 checkstyle 0m 28s hadoop-hdfs-project: The patch generated 1 new + 127 unchanged - 0 fixed = 128 total (was 127) +1 mvnsite 1m 22s the patch passed +1 mvneclipse 0m 19s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 27s the patch passed +1 javadoc 1m 12s the patch passed +1 unit 0m 53s hadoop-hdfs-client in the patch passed. -1 unit 58m 44s hadoop-hdfs in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 86m 33s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics   hadoop.cli.TestHDFSCLI   hadoop.hdfs.server.datanode.TestDataNodeLifeline Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12819178/HDFS-10662.patch JIRA Issue HDFS-10662 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux d785967c9ca1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d84ab8a Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16198/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16198/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16198/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16198/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          daryn Daryn Sharp added a comment -

          Test failures are unrelated. This is just about conversions.

          Show
          daryn Daryn Sharp added a comment - Test failures are unrelated. This is just about conversions.
          Hide
          kihwal Kihwal Lee added a comment -

          +1 lgtm.

          Show
          kihwal Kihwal Lee added a comment - +1 lgtm.
          Hide
          kihwal Kihwal Lee added a comment -

          Unfortunately, this just went in and the patch no longer applies.

          commit a5fb298e56220a35d61b8d2bda716d8fb8ef8bb7
          Author: Akira Ajisaka <aajisaka@apache.org>
          Date:   Tue Aug 2 17:07:59 2016 +0900
          
              HDFS-10707. Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets. Contributed by Vincent Poon.
          
          Show
          kihwal Kihwal Lee added a comment - Unfortunately, this just went in and the patch no longer applies. commit a5fb298e56220a35d61b8d2bda716d8fb8ef8bb7 Author: Akira Ajisaka <aajisaka@apache.org> Date: Tue Aug 2 17:07:59 2016 +0900 HDFS-10707. Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets. Contributed by Vincent Poon.
          Hide
          daryn Daryn Sharp added a comment -

          resolved minor conflicts.

          Show
          daryn Daryn Sharp added a comment - resolved minor conflicts.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 12s Docker mode activated.
          0 patch 0m 1s The patch file was not named according to hadoop's naming conventions. Please see https://wiki.apache.org/hadoop/HowToContribute for instructions.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          0 mvndep 0m 31s Maven dependency ordering for branch
          +1 mvninstall 7m 8s trunk passed
          +1 compile 1m 25s trunk passed
          +1 checkstyle 0m 32s trunk passed
          +1 mvnsite 1m 25s trunk passed
          +1 mvneclipse 0m 24s trunk passed
          +1 findbugs 3m 6s trunk passed
          +1 javadoc 1m 14s trunk passed
          0 mvndep 0m 6s Maven dependency ordering for patch
          +1 mvninstall 1m 17s the patch passed
          +1 compile 1m 20s the patch passed
          +1 javac 1m 20s the patch passed
          +1 checkstyle 0m 28s the patch passed
          +1 mvnsite 1m 21s the patch passed
          +1 mvneclipse 0m 20s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 3m 17s the patch passed
          +1 javadoc 1m 18s the patch passed
          +1 unit 1m 1s hadoop-hdfs-client in the patch passed.
          -1 unit 65m 11s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 22s The patch does not generate ASF License warnings.
          93m 28s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.namenode.TestDecommissioningStatus
            hadoop.hdfs.server.namenode.TestEditLog
            hadoop.hdfs.server.blockmanagement.TestBlockManager



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821862/HDFS-10662.patch.1
          JIRA Issue HDFS-10662
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 02e9740e6eae 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 2d82276
          Default Java 1.8.0_101
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16309/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16309/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16309/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated. 0 patch 0m 1s The patch file was not named according to hadoop's naming conventions. Please see https://wiki.apache.org/hadoop/HowToContribute for instructions. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 31s Maven dependency ordering for branch +1 mvninstall 7m 8s trunk passed +1 compile 1m 25s trunk passed +1 checkstyle 0m 32s trunk passed +1 mvnsite 1m 25s trunk passed +1 mvneclipse 0m 24s trunk passed +1 findbugs 3m 6s trunk passed +1 javadoc 1m 14s trunk passed 0 mvndep 0m 6s Maven dependency ordering for patch +1 mvninstall 1m 17s the patch passed +1 compile 1m 20s the patch passed +1 javac 1m 20s the patch passed +1 checkstyle 0m 28s the patch passed +1 mvnsite 1m 21s the patch passed +1 mvneclipse 0m 20s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 17s the patch passed +1 javadoc 1m 18s the patch passed +1 unit 1m 1s hadoop-hdfs-client in the patch passed. -1 unit 65m 11s hadoop-hdfs in the patch failed. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 93m 28s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestDecommissioningStatus   hadoop.hdfs.server.namenode.TestEditLog   hadoop.hdfs.server.blockmanagement.TestBlockManager Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821862/HDFS-10662.patch.1 JIRA Issue HDFS-10662 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 02e9740e6eae 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 2d82276 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16309/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16309/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16309/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          daryn Daryn Sharp added a comment -

          Test failures are unrelated.

          Show
          daryn Daryn Sharp added a comment - Test failures are unrelated.
          Hide
          kihwal Kihwal Lee added a comment -

          TestEditLog failure is being fixed in HDFS-10169/HDFS-10722.

          Show
          kihwal Kihwal Lee added a comment - TestEditLog failure is being fixed in HDFS-10169 / HDFS-10722 .
          Hide
          kihwal Kihwal Lee added a comment -

          +1 on the latest patch.

          Show
          kihwal Kihwal Lee added a comment - +1 on the latest patch.
          Hide
          kihwal Kihwal Lee added a comment -

          Committed it to trunk, but for branch-2, we have to wait for HDFS-10707.

          Show
          kihwal Kihwal Lee added a comment - Committed it to trunk, but for branch-2, we have to wait for HDFS-10707 .
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #10214 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10214/)
          HDFS-10662. Optimize UTF8 string/byte conversions. Contributed by Daryn (kihwal: rev 6ae39199dac6ac7be6802b31452552c76da16e24)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirXAttrOp.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirMkdirOp.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSUtilClient.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #10214 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10214/ ) HDFS-10662 . Optimize UTF8 string/byte conversions. Contributed by Daryn (kihwal: rev 6ae39199dac6ac7be6802b31452552c76da16e24) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirXAttrOp.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirMkdirOp.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSUtilClient.java
          Hide
          kihwal Kihwal Lee added a comment -

          Picked to branch-2 and branch-2.8.

          Show
          kihwal Kihwal Lee added a comment - Picked to branch-2 and branch-2.8.
          Hide
          zhz Zhe Zhang added a comment -

          Sorry to reopen the JIRA. I'm backporting to branch-2.7 and it was quite messy.

          Daryn Sharp Kihwal Lee I'd appreciate it if you could take a look.

          Show
          zhz Zhe Zhang added a comment - Sorry to reopen the JIRA. I'm backporting to branch-2.7 and it was quite messy. Daryn Sharp Kihwal Lee I'd appreciate it if you could take a look.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 11m 27s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 6m 18s branch-2.7 passed
          +1 compile 1m 1s branch-2.7 passed with JDK v1.8.0_101
          +1 compile 1m 4s branch-2.7 passed with JDK v1.7.0_111
          +1 checkstyle 0m 26s branch-2.7 passed
          +1 mvnsite 0m 58s branch-2.7 passed
          +1 mvneclipse 0m 14s branch-2.7 passed
          +1 findbugs 2m 59s branch-2.7 passed
          +1 javadoc 0m 59s branch-2.7 passed with JDK v1.8.0_101
          +1 javadoc 1m 46s branch-2.7 passed with JDK v1.7.0_111
          +1 mvninstall 0m 54s the patch passed
          +1 compile 0m 56s the patch passed with JDK v1.8.0_101
          +1 javac 0m 56s the patch passed
          +1 compile 0m 59s the patch passed with JDK v1.7.0_111
          +1 javac 0m 59s the patch passed
          -0 checkstyle 0m 23s hadoop-hdfs-project/hadoop-hdfs: The patch generated 3 new + 200 unchanged - 0 fixed = 203 total (was 200)
          +1 mvnsite 0m 57s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          -1 whitespace 0m 0s The patch has 1946 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          -1 whitespace 0m 44s The patch 84 line(s) with tabs.
          +1 findbugs 3m 8s the patch passed
          +1 javadoc 0m 56s the patch passed with JDK v1.8.0_101
          +1 javadoc 1m 42s the patch passed with JDK v1.7.0_111
          -1 unit 45m 48s hadoop-hdfs in the patch failed with JDK v1.7.0_111.
          -1 asflicense 0m 18s The patch generated 3 ASF License warnings.
          133m 49s



          Reason Tests
          JDK v1.8.0_101 Failed junit tests hadoop.hdfs.server.datanode.TestBlockReplacement
            hadoop.hdfs.web.TestWebHdfsTimeouts
            hadoop.hdfs.server.balancer.TestBalancer
            hadoop.hdfs.server.namenode.ha.TestDFSZKFailoverController
            hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA
            hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
          JDK v1.7.0_111 Failed junit tests hadoop.hdfs.server.balancer.TestBalancer
            hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:c420dfe
          JIRA Issue HDFS-10662
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12826529/HDFS-10662-branch-2.7.00.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 850d04c25d2c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2.7 / 73ba5a0
          Default Java 1.7.0_111
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16601/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16601/artifact/patchprocess/whitespace-eol.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16601/artifact/patchprocess/whitespace-tabs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16601/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_111.txt
          JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16601/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/16601/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16601/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 11m 27s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 18s branch-2.7 passed +1 compile 1m 1s branch-2.7 passed with JDK v1.8.0_101 +1 compile 1m 4s branch-2.7 passed with JDK v1.7.0_111 +1 checkstyle 0m 26s branch-2.7 passed +1 mvnsite 0m 58s branch-2.7 passed +1 mvneclipse 0m 14s branch-2.7 passed +1 findbugs 2m 59s branch-2.7 passed +1 javadoc 0m 59s branch-2.7 passed with JDK v1.8.0_101 +1 javadoc 1m 46s branch-2.7 passed with JDK v1.7.0_111 +1 mvninstall 0m 54s the patch passed +1 compile 0m 56s the patch passed with JDK v1.8.0_101 +1 javac 0m 56s the patch passed +1 compile 0m 59s the patch passed with JDK v1.7.0_111 +1 javac 0m 59s the patch passed -0 checkstyle 0m 23s hadoop-hdfs-project/hadoop-hdfs: The patch generated 3 new + 200 unchanged - 0 fixed = 203 total (was 200) +1 mvnsite 0m 57s the patch passed +1 mvneclipse 0m 13s the patch passed -1 whitespace 0m 0s The patch has 1946 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply -1 whitespace 0m 44s The patch 84 line(s) with tabs. +1 findbugs 3m 8s the patch passed +1 javadoc 0m 56s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 42s the patch passed with JDK v1.7.0_111 -1 unit 45m 48s hadoop-hdfs in the patch failed with JDK v1.7.0_111. -1 asflicense 0m 18s The patch generated 3 ASF License warnings. 133m 49s Reason Tests JDK v1.8.0_101 Failed junit tests hadoop.hdfs.server.datanode.TestBlockReplacement   hadoop.hdfs.web.TestWebHdfsTimeouts   hadoop.hdfs.server.balancer.TestBalancer   hadoop.hdfs.server.namenode.ha.TestDFSZKFailoverController   hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots JDK v1.7.0_111 Failed junit tests hadoop.hdfs.server.balancer.TestBalancer   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots Subsystem Report/Notes Docker Image:yetus/hadoop:c420dfe JIRA Issue HDFS-10662 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12826529/HDFS-10662-branch-2.7.00.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 850d04c25d2c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / 73ba5a0 Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16601/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16601/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16601/artifact/patchprocess/whitespace-tabs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16601/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_111.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16601/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/16601/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16601/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          kihwal Kihwal Lee added a comment -

          Zhe Zhang, I will take a look. The 2.7 precommit has more test failures because of smaller docker container being used compared to the trunk one. They often fail due to OOM.

          Show
          kihwal Kihwal Lee added a comment - Zhe Zhang , I will take a look. The 2.7 precommit has more test failures because of smaller docker container being used compared to the trunk one. They often fail due to OOM.
          Hide
          kihwal Kihwal Lee added a comment -

          Zhe Zhang, the patch looks good in general. There is an unused import (*.Charsets) to be removed from DFSUtil, FSDirMkdirOp and FSDirStatAndListingOp. Daryn's original patch did that, but then HDFS-10707 got committed, so the later patch didn't have to.

          Show
          kihwal Kihwal Lee added a comment - Zhe Zhang , the patch looks good in general. There is an unused import (*.Charsets) to be removed from DFSUtil, FSDirMkdirOp and FSDirStatAndListingOp. Daryn's original patch did that, but then HDFS-10707 got committed, so the later patch didn't have to.
          Hide
          zhz Zhe Zhang added a comment -

          Thanks Kihwal Lee! I removed the unused import and verified all reported test failures. Just pushed the commit to branch-2.7.

          Show
          zhz Zhe Zhang added a comment - Thanks Kihwal Lee ! I removed the unused import and verified all reported test failures. Just pushed the commit to branch-2.7.

            People

            • Assignee:
              daryn Daryn Sharp
              Reporter:
              daryn Daryn Sharp
            • Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development