Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-5177

blocksScheduled count should be decremented for abandoned blocks

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.1.0-beta, 3.0.0-alpha1
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      DatanodeDescriptor#incBlocksScheduled() will be called for all datanodes of the block on each allocation. But same should be decremented for abandoned blocks.

      When one of the datanodes is down and same is allocated for the block along with other live datanodes, then this block will be abandoned, but the scheduled count on other datanodes will consider live datanodes as loaded, but in reality these datanodes may not be loaded.

      Anyway this scheduled count will be rolled every 20 mins.

      Problem will come if the rate of creation of files is more. Due to increase in the scheduled count, there might be chances of missing local datanode to write to. and some times writes also can fail in small clusters.

      So we need to decrement the unnecessary count on abandon block call.

      1. HDFS-5177.patch
        13 kB
        Vinayakumar B
      2. HDFS-5177.patch
        13 kB
        Vinayakumar B
      3. HDFS-5177.patch
        14 kB
        Vinayakumar B
      4. HDFS-5177-04.patch
        6 kB
        Vinayakumar B
      5. HDFS-5177-05.patch
        5 kB
        Vinayakumar B

        Activity

        Hide
        vinayrpet Vinayakumar B added a comment -

        Attaching the patch.
        Please review

        Show
        vinayrpet Vinayakumar B added a comment - Attaching the patch. Please review
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12602299/HDFS-5177.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
        org.apache.hadoop.hdfs.TestDFSShell

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4946//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4946//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12602299/HDFS-5177.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA org.apache.hadoop.hdfs.TestDFSShell +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/4946//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/4946//console This message is automatically generated.
        Hide
        vinayrpet Vinayakumar B added a comment -

        Attaching the rebased patch.

        Please some one can review this patch. thanks

        Show
        vinayrpet Vinayakumar B added a comment - Attaching the rebased patch. Please some one can review this patch. thanks
        Hide
        hadoopqa Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12614577/HDFS-5177.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/5482//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5482//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12614577/HDFS-5177.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/5482//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5482//console This message is automatically generated.
        Hide
        vinayrpet Vinayakumar B added a comment -

        Updated latest patch.
        Please review

        Show
        vinayrpet Vinayakumar B added a comment - Updated latest patch. Please review
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12616329/HDFS-5177.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup
        org.apache.hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/5602//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5602//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12616329/HDFS-5177.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup org.apache.hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/5602//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/5602//console This message is automatically generated.
        Hide
        vinayrpet Vinayakumar B added a comment -

        Hi all,
        Can someone please take a look at the patch.
        Thanks in advance.

        Show
        vinayrpet Vinayakumar B added a comment - Hi all, Can someone please take a look at the patch. Thanks in advance.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 patch 0m 0s The patch command could not apply the patch during dryrun.



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12616329/HDFS-5177.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / f1a152c
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10590/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 0s The patch command could not apply the patch during dryrun. Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12616329/HDFS-5177.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / f1a152c Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10590/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 patch 0m 0s The patch command could not apply the patch during dryrun.



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12616329/HDFS-5177.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / f1a152c
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10616/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 0s The patch command could not apply the patch during dryrun. Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12616329/HDFS-5177.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / f1a152c Console output https://builds.apache.org/job/PreCommit-HDFS-Build/10616/console This message was automatically generated.
        Hide
        vinayrpet Vinayakumar B added a comment -

        Updated the patch.
        No change in protocol in latest patch. (Previous patches changed the clientprotocol).

        Please review.

        Show
        vinayrpet Vinayakumar B added a comment - Updated the patch. No change in protocol in latest patch. (Previous patches changed the clientprotocol). Please review.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        -1 mvninstall 3m 4s root in trunk failed.
        -1 compile 0m 37s hadoop-hdfs in trunk failed with JDK v1.8.0_74.
        -1 compile 0m 27s hadoop-hdfs in trunk failed with JDK v1.7.0_95.
        +1 checkstyle 0m 21s trunk passed
        -1 mvnsite 0m 28s hadoop-hdfs in trunk failed.
        +1 mvneclipse 0m 14s trunk passed
        -1 findbugs 0m 24s hadoop-hdfs in trunk failed.
        +1 javadoc 1m 14s trunk passed with JDK v1.8.0_74
        +1 javadoc 2m 1s trunk passed with JDK v1.7.0_95
        -1 mvninstall 0m 24s hadoop-hdfs in the patch failed.
        -1 compile 0m 31s hadoop-hdfs in the patch failed with JDK v1.8.0_74.
        -1 javac 0m 31s hadoop-hdfs in the patch failed with JDK v1.8.0_74.
        -1 compile 0m 30s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
        -1 javac 0m 30s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
        +1 checkstyle 0m 19s the patch passed
        -1 mvnsite 0m 27s hadoop-hdfs in the patch failed.
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        -1 findbugs 0m 28s hadoop-hdfs in the patch failed.
        +1 javadoc 1m 22s the patch passed with JDK v1.8.0_74
        +1 javadoc 2m 8s the patch passed with JDK v1.7.0_95
        -1 unit 0m 41s hadoop-hdfs in the patch failed with JDK v1.8.0_74.
        -1 unit 0m 26s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
        -1 asflicense 0m 18s Patch generated 1 ASF License warnings.
        18m 55s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:fbe3e86
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12795507/HDFS-5177-04.patch
        JIRA Issue HDFS-5177
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux c21667e765dd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / e5ff0ea
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/branch-mvninstall-root.txt
        compile https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/branch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt
        compile https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/branch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
        mvnsite https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/branch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt
        mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt
        compile https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt
        javac https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt
        compile https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
        javac https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
        mvnsite https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14948/testReport/
        asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14948/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. -1 mvninstall 3m 4s root in trunk failed. -1 compile 0m 37s hadoop-hdfs in trunk failed with JDK v1.8.0_74. -1 compile 0m 27s hadoop-hdfs in trunk failed with JDK v1.7.0_95. +1 checkstyle 0m 21s trunk passed -1 mvnsite 0m 28s hadoop-hdfs in trunk failed. +1 mvneclipse 0m 14s trunk passed -1 findbugs 0m 24s hadoop-hdfs in trunk failed. +1 javadoc 1m 14s trunk passed with JDK v1.8.0_74 +1 javadoc 2m 1s trunk passed with JDK v1.7.0_95 -1 mvninstall 0m 24s hadoop-hdfs in the patch failed. -1 compile 0m 31s hadoop-hdfs in the patch failed with JDK v1.8.0_74. -1 javac 0m 31s hadoop-hdfs in the patch failed with JDK v1.8.0_74. -1 compile 0m 30s hadoop-hdfs in the patch failed with JDK v1.7.0_95. -1 javac 0m 30s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 checkstyle 0m 19s the patch passed -1 mvnsite 0m 27s hadoop-hdfs in the patch failed. +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. -1 findbugs 0m 28s hadoop-hdfs in the patch failed. +1 javadoc 1m 22s the patch passed with JDK v1.8.0_74 +1 javadoc 2m 8s the patch passed with JDK v1.7.0_95 -1 unit 0m 41s hadoop-hdfs in the patch failed with JDK v1.8.0_74. -1 unit 0m 26s hadoop-hdfs in the patch failed with JDK v1.7.0_95. -1 asflicense 0m 18s Patch generated 1 ASF License warnings. 18m 55s Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12795507/HDFS-5177-04.patch JIRA Issue HDFS-5177 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c21667e765dd 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e5ff0ea Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/branch-mvninstall-root.txt compile https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/branch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt compile https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/branch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt mvnsite https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/branch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt compile https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt javac https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt compile https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt javac https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/patch-compile-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt mvnsite https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/patch-findbugs-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14948/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/14948/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14948/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        vinayrpet Vinayakumar B added a comment -

        Compilation error was from HDFS-9694's broken commit.
        Now thats fixed. Triggering one more jenkins run,

        Show
        vinayrpet Vinayakumar B added a comment - Compilation error was from HDFS-9694 's broken commit. Now thats fixed. Triggering one more jenkins run,
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 16s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 53s trunk passed
        +1 compile 0m 58s trunk passed with JDK v1.8.0_74
        +1 compile 0m 48s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 24s trunk passed
        +1 mvnsite 1m 0s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 2m 7s trunk passed
        +1 javadoc 1m 21s trunk passed with JDK v1.8.0_74
        +1 javadoc 2m 7s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 53s the patch passed
        +1 compile 0m 54s the patch passed with JDK v1.8.0_74
        +1 javac 0m 54s the patch passed
        +1 compile 0m 45s the patch passed with JDK v1.7.0_95
        +1 javac 0m 45s the patch passed
        +1 checkstyle 0m 20s the patch passed
        +1 mvnsite 0m 53s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 17s the patch passed
        +1 javadoc 1m 19s the patch passed with JDK v1.8.0_74
        +1 javadoc 1m 56s the patch passed with JDK v1.7.0_95
        -1 unit 88m 42s hadoop-hdfs in the patch failed with JDK v1.8.0_74.
        -1 unit 82m 11s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
        -1 asflicense 0m 21s Patch generated 1 ASF License warnings.
        200m 42s



        Reason Tests
        JDK v1.8.0_74 Failed junit tests hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl
        JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
          hadoop.hdfs.server.namenode.TestEditLog



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:fbe3e86
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12795507/HDFS-5177-04.patch
        JIRA Issue HDFS-5177
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux b52ef63d9a06 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 49ff54c
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14952/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14952/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14952/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14952/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14952/testReport/
        asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/14952/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14952/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 53s trunk passed +1 compile 0m 58s trunk passed with JDK v1.8.0_74 +1 compile 0m 48s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 24s trunk passed +1 mvnsite 1m 0s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 2m 7s trunk passed +1 javadoc 1m 21s trunk passed with JDK v1.8.0_74 +1 javadoc 2m 7s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 53s the patch passed +1 compile 0m 54s the patch passed with JDK v1.8.0_74 +1 javac 0m 54s the patch passed +1 compile 0m 45s the patch passed with JDK v1.7.0_95 +1 javac 0m 45s the patch passed +1 checkstyle 0m 20s the patch passed +1 mvnsite 0m 53s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 17s the patch passed +1 javadoc 1m 19s the patch passed with JDK v1.8.0_74 +1 javadoc 1m 56s the patch passed with JDK v1.7.0_95 -1 unit 88m 42s hadoop-hdfs in the patch failed with JDK v1.8.0_74. -1 unit 82m 11s hadoop-hdfs in the patch failed with JDK v1.7.0_95. -1 asflicense 0m 21s Patch generated 1 ASF License warnings. 200m 42s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.namenode.TestEditLog Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12795507/HDFS-5177-04.patch JIRA Issue HDFS-5177 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b52ef63d9a06 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 49ff54c Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/14952/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14952/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14952/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14952/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14952/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/14952/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14952/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        vinayrpet Vinayakumar B added a comment -

        Test failures are not related.

        Show
        vinayrpet Vinayakumar B added a comment - Test failures are not related.
        Hide
        andrew.wang Andrew Wang added a comment -

        Patch looks good, thanks for working on this Vinay. One question, do we need to do the LOG-related changes to turn debug logging in the test?

        Otherwise +1.

        Show
        andrew.wang Andrew Wang added a comment - Patch looks good, thanks for working on this Vinay. One question, do we need to do the LOG-related changes to turn debug logging in the test? Otherwise +1.
        Hide
        vinayrpet Vinayakumar B added a comment -

        I have reverted the LOG related changes.

        Thanks Andrew Wang, I will commit it based on your +1, once the jenkins says, Go.

        Show
        vinayrpet Vinayakumar B added a comment - I have reverted the LOG related changes. Thanks Andrew Wang , I will commit it based on your +1, once the jenkins says, Go.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 33s trunk passed
        +1 compile 0m 37s trunk passed with JDK v1.8.0_74
        +1 compile 0m 41s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 21s trunk passed
        +1 mvnsite 0m 51s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 56s trunk passed
        +1 javadoc 1m 4s trunk passed with JDK v1.8.0_74
        +1 javadoc 1m 49s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 47s the patch passed
        +1 compile 0m 38s the patch passed with JDK v1.8.0_74
        +1 javac 0m 38s the patch passed
        +1 compile 0m 41s the patch passed with JDK v1.7.0_95
        +1 javac 0m 41s the patch passed
        +1 checkstyle 0m 19s the patch passed
        +1 mvnsite 0m 49s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 6s the patch passed
        +1 javadoc 1m 4s the patch passed with JDK v1.8.0_74
        +1 javadoc 1m 44s the patch passed with JDK v1.7.0_95
        -1 unit 70m 49s hadoop-hdfs in the patch failed with JDK v1.8.0_74.
        -1 unit 66m 57s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
        -1 asflicense 0m 21s Patch generated 1 ASF License warnings.
        162m 56s



        Reason Tests
        JDK v1.8.0_74 Failed junit tests hadoop.hdfs.TestHFlush
          hadoop.hdfs.server.balancer.TestBalancer
        JDK v1.7.0_95 Failed junit tests hadoop.hdfs.TestHFlush
          hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency
          hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:fbe3e86
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12795992/HDFS-5177-05.patch
        JIRA Issue HDFS-5177
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux e55d5cfa7493 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 8286270
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14992/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/14992/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14992/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14992/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14992/testReport/
        asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/14992/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14992/console
        Powered by Apache Yetus 0.2.0 http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 33s trunk passed +1 compile 0m 37s trunk passed with JDK v1.8.0_74 +1 compile 0m 41s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 21s trunk passed +1 mvnsite 0m 51s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 56s trunk passed +1 javadoc 1m 4s trunk passed with JDK v1.8.0_74 +1 javadoc 1m 49s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 47s the patch passed +1 compile 0m 38s the patch passed with JDK v1.8.0_74 +1 javac 0m 38s the patch passed +1 compile 0m 41s the patch passed with JDK v1.7.0_95 +1 javac 0m 41s the patch passed +1 checkstyle 0m 19s the patch passed +1 mvnsite 0m 49s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 6s the patch passed +1 javadoc 1m 4s the patch passed with JDK v1.8.0_74 +1 javadoc 1m 44s the patch passed with JDK v1.7.0_95 -1 unit 70m 49s hadoop-hdfs in the patch failed with JDK v1.8.0_74. -1 unit 66m 57s hadoop-hdfs in the patch failed with JDK v1.7.0_95. -1 asflicense 0m 21s Patch generated 1 ASF License warnings. 162m 56s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.hdfs.TestHFlush   hadoop.hdfs.server.balancer.TestBalancer JDK v1.7.0_95 Failed junit tests hadoop.hdfs.TestHFlush   hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency   hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12795992/HDFS-5177-05.patch JIRA Issue HDFS-5177 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e55d5cfa7493 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 8286270 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/14992/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14992/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14992/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14992/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14992/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/14992/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14992/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
        Hide
        vinayrpet Vinayakumar B added a comment -

        Tests are unrelated.

        Show
        vinayrpet Vinayakumar B added a comment - Tests are unrelated.
        Hide
        vinayrpet Vinayakumar B added a comment -

        Committed to trunk, branch-2 and branch-2.8
        Thanks Andrew Wang for the review.

        Show
        vinayrpet Vinayakumar B added a comment - Committed to trunk, branch-2 and branch-2.8 Thanks Andrew Wang for the review.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #9526 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9526/)
        HDFS-5177. blocksScheduled count should be decremented for abandoned (vinayakumarb: rev 09d63d5a192b5d6b172f94ff6c94da348fd49ea6)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeStorageInfo.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlocksScheduledCounter.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9526 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9526/ ) HDFS-5177 . blocksScheduled count should be decremented for abandoned (vinayakumarb: rev 09d63d5a192b5d6b172f94ff6c94da348fd49ea6) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeStorageInfo.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlocksScheduledCounter.java

          People

          • Assignee:
            vinayrpet Vinayakumar B
            Reporter:
            vinayrpet Vinayakumar B
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development