Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10691

FileDistribution fails in hdfs oiv command due to ArrayIndexOutOfBoundsException

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.1
    • Fix Version/s: 2.8.0, 2.7.4, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      I use hdfs oiv -p FileDistribution command to do a file analyse. But the ArrayIndexOutOfBoundsException happened and lead the process terminated. The stack infos:

      Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 103
      	at org.apache.hadoop.hdfs.tools.offlineImageViewer.FileDistributionCalculator.run(FileDistributionCalculator.java:243)
      	at org.apache.hadoop.hdfs.tools.offlineImageViewer.FileDistributionCalculator.visit(FileDistributionCalculator.java:176)
      	at org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageViewerPB.run(OfflineImageViewerPB.java:176)
      	at org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageViewerPB.main(OfflineImageViewerPB.java:129)
      

      I looked into the code and I found the exception was threw in increasing count of distribution. And the reason for the exception is that the bucket number was more than the distribution's length.

      Here are my steps:
      1).The input command params:

      hdfs oiv -p FileDistribution -maxSize 104857600 -step 1024000
      

      The numIntervals in code should be 104857600/1024000 =102(real value:102.4), so the distribution's length should be numIntervals + 1 = 103.
      2).The ArrayIndexOutOfBoundsException will happens when the file size is in range ((maxSize/step)*step, maxSize]. For example, if the size of one file is 104800000, and it's in range of size as mentioned before. And the bucket number is calculated as 104800000/1024000=102.3, then in code we do the Math.ceil of this, so the final value should be 103. But the distribution's length is also 103, it means the index is from 0 to 102. So the ArrayIndexOutOfBoundsException happens.

      In a word, the exception will happens when maxSize can not be divided by step and meanwhile the size of file is in range ((maxSize/step)*step, maxSize]. The related logic should be changed from

      int bucket = fileSize > maxSize ? distribution.length - 1 : (int) Math
                  .ceil((double)fileSize / steps);
      

      to

      int bucket =
                  fileSize >= maxSize || fileSize > (maxSize / steps) * steps ?
                      distribution.length - 1 : (int) Math.ceil((double) fileSize / steps);
      
      1. HDFS-10691.001.patch
        2 kB
        Yiqun Lin
      2. HDFS-10691.002.patch
        5 kB
        Yiqun Lin
      3. HDFS-10691.003.patch
        5 kB
        Yiqun Lin
      4. HDFS-10691-branch-2.7.patch
        5 kB
        Yiqun Lin

        Activity

        Hide
        linyiqun Yiqun Lin added a comment -

        Attach a initial patch, thanks for review.

        Show
        linyiqun Yiqun Lin added a comment - Attach a initial patch, thanks for review.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 7m 0s trunk passed
        +1 compile 0m 46s trunk passed
        +1 checkstyle 0m 26s trunk passed
        +1 mvnsite 0m 52s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 1m 47s trunk passed
        +1 javadoc 0m 54s trunk passed
        +1 mvninstall 0m 46s the patch passed
        +1 compile 0m 44s the patch passed
        +1 javac 0m 44s the patch passed
        -0 checkstyle 0m 23s hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 16 unchanged - 1 fixed = 18 total (was 17)
        +1 mvnsite 0m 49s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 50s the patch passed
        +1 javadoc 0m 51s the patch passed
        +1 unit 57m 23s hadoop-hdfs in the patch passed.
        +1 asflicense 0m 20s The patch does not generate ASF License warnings.
        76m 45s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820130/HDFS-10691.001.patch
        JIRA Issue HDFS-10691
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 93bf7afe5832 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 7cac765
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16188/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16188/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16188/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 0s trunk passed +1 compile 0m 46s trunk passed +1 checkstyle 0m 26s trunk passed +1 mvnsite 0m 52s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 47s trunk passed +1 javadoc 0m 54s trunk passed +1 mvninstall 0m 46s the patch passed +1 compile 0m 44s the patch passed +1 javac 0m 44s the patch passed -0 checkstyle 0m 23s hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 16 unchanged - 1 fixed = 18 total (was 17) +1 mvnsite 0m 49s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 50s the patch passed +1 javadoc 0m 51s the patch passed +1 unit 57m 23s hadoop-hdfs in the patch passed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 76m 45s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820130/HDFS-10691.001.patch JIRA Issue HDFS-10691 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 93bf7afe5832 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 7cac765 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16188/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16188/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16188/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        linyiqun Yiqun Lin added a comment - - edited

        I made a test in my local env, I found the description for this jira seems not correct. The size for here should be in ((maxSize/step)*step, maxSize]. Post the new patch with the new test. Can run the test for this issue, I have tested in my local env, if I didn't apply the code with my patch, the exception will be threw in this test. Thanks for review.

        Show
        linyiqun Yiqun Lin added a comment - - edited I made a test in my local env, I found the description for this jira seems not correct. The size for here should be in ((maxSize/step)*step, maxSize]. Post the new patch with the new test. Can run the test for this issue, I have tested in my local env, if I didn't apply the code with my patch, the exception will be threw in this test. Thanks for review.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 1s trunk passed
        +1 compile 0m 53s trunk passed
        +1 checkstyle 0m 29s trunk passed
        +1 mvnsite 0m 54s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 1m 44s trunk passed
        +1 javadoc 0m 59s trunk passed
        +1 mvninstall 0m 50s the patch passed
        +1 compile 0m 44s the patch passed
        +1 javac 0m 44s the patch passed
        -0 checkstyle 0m 24s hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 43 unchanged - 2 fixed = 47 total (was 45)
        +1 mvnsite 0m 53s the patch passed
        +1 mvneclipse 0m 9s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 53s the patch passed
        +1 javadoc 1m 0s the patch passed
        -1 unit 78m 5s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 21s The patch does not generate ASF License warnings.
        98m 10s



        Reason Tests
        Failed junit tests hadoop.cli.TestHDFSCLI
          hadoop.hdfs.server.namenode.TestNameNodeRespectsBindHostKeys



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820369/HDFS-10691.002.patch
        JIRA Issue HDFS-10691
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 29195ab04100 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 55d5993
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16207/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16207/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16207/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16207/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 1s trunk passed +1 compile 0m 53s trunk passed +1 checkstyle 0m 29s trunk passed +1 mvnsite 0m 54s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 44s trunk passed +1 javadoc 0m 59s trunk passed +1 mvninstall 0m 50s the patch passed +1 compile 0m 44s the patch passed +1 javac 0m 44s the patch passed -0 checkstyle 0m 24s hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 43 unchanged - 2 fixed = 47 total (was 45) +1 mvnsite 0m 53s the patch passed +1 mvneclipse 0m 9s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 53s the patch passed +1 javadoc 1m 0s the patch passed -1 unit 78m 5s hadoop-hdfs in the patch failed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 98m 10s Reason Tests Failed junit tests hadoop.cli.TestHDFSCLI   hadoop.hdfs.server.namenode.TestNameNodeRespectsBindHostKeys Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820369/HDFS-10691.002.patch JIRA Issue HDFS-10691 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 29195ab04100 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 55d5993 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16207/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16207/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16207/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16207/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Thanks Yiqun Lin for reporting this and providing the patches.
        To avoid the exception, I'm thinking the following code is more clear. What do you think?

            int bucket = Math.ceil((double)fileSize / steps);
            if (bucket >= distribution.length) {
              bucket = distribution.length - 1;
            }
        
        Show
        ajisakaa Akira Ajisaka added a comment - Thanks Yiqun Lin for reporting this and providing the patches. To avoid the exception, I'm thinking the following code is more clear. What do you think? int bucket = Math .ceil(( double )fileSize / steps); if (bucket >= distribution.length) { bucket = distribution.length - 1; }
        Hide
        linyiqun Yiqun Lin added a comment -

        Thanks Akira Ajisaka for the review. Your suggestion seems good, but I think it will be better to make a change like this and not to change the orginal logic for calculating the bucket value.

                 int bucket = fileSize > maxSize ? distribution.length - 1 : (int) Math
                     .ceil((double)fileSize / steps);
                if (bucket >= distribution.length) {
                  bucket = distribution.length - 1;
                }
        

        Because if the fileSize is already more than maxSize, it can just return distribution.length - 1 and there is no need to do the (int) Math.ceil((double)fileSize / steps);.

        Post a new patch to simlify the logic of this and fix checkstyle warnings.

        Show
        linyiqun Yiqun Lin added a comment - Thanks Akira Ajisaka for the review. Your suggestion seems good, but I think it will be better to make a change like this and not to change the orginal logic for calculating the bucket value. int bucket = fileSize > maxSize ? distribution.length - 1 : ( int ) Math .ceil(( double )fileSize / steps); if (bucket >= distribution.length) { bucket = distribution.length - 1; } Because if the fileSize is already more than maxSize, it can just return distribution.length - 1 and there is no need to do the (int) Math.ceil((double)fileSize / steps); . Post a new patch to simlify the logic of this and fix checkstyle warnings.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 13s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 35s trunk passed
        +1 compile 0m 46s trunk passed
        +1 checkstyle 0m 25s trunk passed
        +1 mvnsite 0m 50s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 40s trunk passed
        +1 javadoc 0m 56s trunk passed
        +1 mvninstall 0m 45s the patch passed
        +1 compile 0m 41s the patch passed
        +1 javac 0m 41s the patch passed
        +1 checkstyle 0m 23s the patch passed
        +1 mvnsite 0m 49s the patch passed
        +1 mvneclipse 0m 9s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 49s the patch passed
        +1 javadoc 0m 53s the patch passed
        +1 unit 57m 57s hadoop-hdfs in the patch passed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        76m 34s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820686/HDFS-10691.003.patch
        JIRA Issue HDFS-10691
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 03e832ae67e9 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 414fbfa
        Default Java 1.8.0_101
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16235/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16235/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 35s trunk passed +1 compile 0m 46s trunk passed +1 checkstyle 0m 25s trunk passed +1 mvnsite 0m 50s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 40s trunk passed +1 javadoc 0m 56s trunk passed +1 mvninstall 0m 45s the patch passed +1 compile 0m 41s the patch passed +1 javac 0m 41s the patch passed +1 checkstyle 0m 23s the patch passed +1 mvnsite 0m 49s the patch passed +1 mvneclipse 0m 9s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 49s the patch passed +1 javadoc 0m 53s the patch passed +1 unit 57m 57s hadoop-hdfs in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 76m 34s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820686/HDFS-10691.003.patch JIRA Issue HDFS-10691 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 03e832ae67e9 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 414fbfa Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16235/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16235/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        +1, checking this in.

        Show
        ajisakaa Akira Ajisaka added a comment - +1, checking this in.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Thanks Yiqun Lin for updating the patch. Committed this to trunk, branch-2, and branch-2.8.
        Would you provide a patch for branch-2.7? Hint: Now we cannot use try-with-resources statement for MiniDFSCluster in branch-2.7. (HDFS-10287)

        Show
        ajisakaa Akira Ajisaka added a comment - Thanks Yiqun Lin for updating the patch. Committed this to trunk, branch-2, and branch-2.8. Would you provide a patch for branch-2.7? Hint: Now we cannot use try-with-resources statement for MiniDFSCluster in branch-2.7. ( HDFS-10287 )
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #10177 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10177/)
        HDFS-10691. FileDistribution fails in hdfs oiv command due to (aajisaka: rev 204a2055b1b9270ae13ea03b7aeac62b65166efd)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FileDistributionCalculator.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FileDistributionVisitor.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #10177 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10177/ ) HDFS-10691 . FileDistribution fails in hdfs oiv command due to (aajisaka: rev 204a2055b1b9270ae13ea03b7aeac62b65166efd) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FileDistributionCalculator.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/FileDistributionVisitor.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java
        Hide
        linyiqun Yiqun Lin added a comment -

        Thanks Akira Ajisaka for the commit, post a patch for branch-2.7.

        Show
        linyiqun Yiqun Lin added a comment - Thanks Akira Ajisaka for the commit, post a patch for branch-2.7.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 15m 41s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 8m 52s branch-2.7 passed
        +1 compile 0m 58s branch-2.7 passed with JDK v1.8.0_101
        +1 compile 1m 1s branch-2.7 passed with JDK v1.7.0_101
        +1 checkstyle 0m 25s branch-2.7 passed
        +1 mvnsite 1m 1s branch-2.7 passed
        +1 mvneclipse 0m 19s branch-2.7 passed
        +1 findbugs 3m 4s branch-2.7 passed
        +1 javadoc 1m 0s branch-2.7 passed with JDK v1.8.0_101
        +1 javadoc 1m 39s branch-2.7 passed with JDK v1.7.0_101
        +1 mvninstall 0m 53s the patch passed
        +1 compile 0m 59s the patch passed with JDK v1.8.0_101
        +1 javac 0m 59s the patch passed
        +1 compile 1m 5s the patch passed with JDK v1.7.0_101
        +1 javac 1m 5s the patch passed
        +1 checkstyle 0m 22s the patch passed
        +1 mvnsite 1m 1s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        -1 whitespace 0m 0s The patch has 2056 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
        -1 whitespace 0m 58s The patch 80 line(s) with tabs.
        +1 findbugs 3m 12s the patch passed
        +1 javadoc 1m 0s the patch passed with JDK v1.8.0_101
        +1 javadoc 1m 39s the patch passed with JDK v1.7.0_101
        -1 unit 54m 28s hadoop-hdfs in the patch failed with JDK v1.7.0_101.
        -1 asflicense 0m 20s The patch generated 3 ASF License warnings.
        158m 30s



        Reason Tests
        JDK v1.8.0_101 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
          hadoop.hdfs.server.balancer.TestBalancer
          hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes
          hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
        JDK v1.7.0_101 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
          hadoop.hdfs.TestRollingUpgrade
          hadoop.hdfs.TestSafeMode
          hadoop.hdfs.server.datanode.TestBlockReplacement



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:c420dfe
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820939/HDFS-10691-branch-2.7.patch
        JIRA Issue HDFS-10691
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 9d4d8de17b39 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision branch-2.7 / 6cb2e97
        Default Java 1.7.0_101
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101
        findbugs v3.0.0
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16251/artifact/patchprocess/whitespace-eol.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16251/artifact/patchprocess/whitespace-tabs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16251/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_101.txt
        JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16251/testReport/
        asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/16251/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16251/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 15m 41s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 52s branch-2.7 passed +1 compile 0m 58s branch-2.7 passed with JDK v1.8.0_101 +1 compile 1m 1s branch-2.7 passed with JDK v1.7.0_101 +1 checkstyle 0m 25s branch-2.7 passed +1 mvnsite 1m 1s branch-2.7 passed +1 mvneclipse 0m 19s branch-2.7 passed +1 findbugs 3m 4s branch-2.7 passed +1 javadoc 1m 0s branch-2.7 passed with JDK v1.8.0_101 +1 javadoc 1m 39s branch-2.7 passed with JDK v1.7.0_101 +1 mvninstall 0m 53s the patch passed +1 compile 0m 59s the patch passed with JDK v1.8.0_101 +1 javac 0m 59s the patch passed +1 compile 1m 5s the patch passed with JDK v1.7.0_101 +1 javac 1m 5s the patch passed +1 checkstyle 0m 22s the patch passed +1 mvnsite 1m 1s the patch passed +1 mvneclipse 0m 14s the patch passed -1 whitespace 0m 0s The patch has 2056 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply -1 whitespace 0m 58s The patch 80 line(s) with tabs. +1 findbugs 3m 12s the patch passed +1 javadoc 1m 0s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 39s the patch passed with JDK v1.7.0_101 -1 unit 54m 28s hadoop-hdfs in the patch failed with JDK v1.7.0_101. -1 asflicense 0m 20s The patch generated 3 ASF License warnings. 158m 30s Reason Tests JDK v1.8.0_101 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   hadoop.hdfs.server.balancer.TestBalancer   hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes   hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot JDK v1.7.0_101 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   hadoop.hdfs.TestRollingUpgrade   hadoop.hdfs.TestSafeMode   hadoop.hdfs.server.datanode.TestBlockReplacement Subsystem Report/Notes Docker Image:yetus/hadoop:c420dfe JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12820939/HDFS-10691-branch-2.7.patch JIRA Issue HDFS-10691 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 9d4d8de17b39 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / 6cb2e97 Default Java 1.7.0_101 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 findbugs v3.0.0 whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16251/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16251/artifact/patchprocess/whitespace-tabs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16251/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_101.txt JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16251/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/16251/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16251/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        +1.

        Show
        ajisakaa Akira Ajisaka added a comment - +1.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Committed this to branch-2.7. Thank you again!

        Show
        ajisakaa Akira Ajisaka added a comment - Committed this to branch-2.7. Thank you again!

          People

          • Assignee:
            linyiqun Yiqun Lin
            Reporter:
            linyiqun Yiqun Lin
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development