Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11295

Check storage remaining instead of node remaining in BlockPlacementPolicyDefault.chooseReplicaToDelete()

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.1
    • Fix Version/s: 2.9.0, 3.0.0-alpha4
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently in BlockPlacementPolicyDefault.chooseReplicaToDelete() the logic for choosing replica to delete is to pick the node with the least free space(node.getRemaining()), if all hearbeats are within the tolerable heartbeat interval.
      However, a node may have multiple storages and node.getRemaining() is a sum of the remainings of them, if free space of the storage with the block to be delete is low, free space of the node could still be high due to other storages of the node, finally the storage chosen may not be the storage with least free space.
      So using storage.getRemaining() to choose a storage with least free space for choosing replica to delete may be a better way to balance storage usage.

      1. HDFS-11295.001.patch
        4 kB
        Elek, Marton
      2. HDFS-11295.002.patch
        7 kB
        Elek, Marton
      3. HDFS-11295.003.patch
        7 kB
        Elek, Marton
      4. HDFS-11295.jpg
        123 kB
        Elek, Marton
      5. HDFS-11295.004.patch
        7 kB
        Elek, Marton

        Activity

        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #11304 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11304/)
        HDFS-11295. Check storage remaining instead of node remaining in (arp: rev d2b3ba9b8fb76753fa1b51661dacbde74aa5c6df)

        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeStorageInfo.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #11304 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11304/ ) HDFS-11295 . Check storage remaining instead of node remaining in (arp: rev d2b3ba9b8fb76753fa1b51661dacbde74aa5c6df) (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeStorageInfo.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        Committed to trunk and branch-2. Thanks for the contribution Elek, Marton.

        Show
        arpitagarwal Arpit Agarwal added a comment - Committed to trunk and branch-2. Thanks for the contribution Elek, Marton .
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        +1 for the v4 patch. The test failures are unrelated.

        Show
        arpitagarwal Arpit Agarwal added a comment - +1 for the v4 patch. The test failures are unrelated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 28s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 17m 30s trunk passed
        +1 compile 1m 0s trunk passed
        +1 checkstyle 0m 40s trunk passed
        +1 mvnsite 1m 11s trunk passed
        +1 mvneclipse 0m 19s trunk passed
        +1 findbugs 1m 58s trunk passed
        +1 javadoc 0m 52s trunk passed
        +1 mvninstall 1m 1s the patch passed
        +1 compile 0m 59s the patch passed
        +1 javac 0m 59s the patch passed
        +1 checkstyle 0m 37s the patch passed
        +1 mvnsite 1m 5s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 2m 8s the patch passed
        +1 javadoc 0m 39s the patch passed
        -1 unit 92m 2s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 21s The patch does not generate ASF License warnings.
        124m 24s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11295
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12854443/HDFS-11295.004.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux fe7d2afe59f7 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 132f758
        Default Java 1.8.0_121
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18436/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18436/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18436/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 28s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 17m 30s trunk passed +1 compile 1m 0s trunk passed +1 checkstyle 0m 40s trunk passed +1 mvnsite 1m 11s trunk passed +1 mvneclipse 0m 19s trunk passed +1 findbugs 1m 58s trunk passed +1 javadoc 0m 52s trunk passed +1 mvninstall 1m 1s the patch passed +1 compile 0m 59s the patch passed +1 javac 0m 59s the patch passed +1 checkstyle 0m 37s the patch passed +1 mvnsite 1m 5s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 8s the patch passed +1 javadoc 0m 39s the patch passed -1 unit 92m 2s hadoop-hdfs in the patch failed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 124m 24s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11295 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12854443/HDFS-11295.004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux fe7d2afe59f7 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 132f758 Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18436/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18436/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18436/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        elek Elek, Marton added a comment -

        Thanks, the review. Improved according to the comments.

        Show
        elek Elek, Marton added a comment - Thanks, the review. Improved according to the comments.
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        Thanks for the visualization Elek, Marton. This makes sense and the change looks good. Let's rename DatanodeStorageInfo#setRemaining to setRemainingForTests and make it package-private.

        A minor concern is the NameNode's mapping of (storage -> block replicas) can be temporarily incorrect. In a healthy cluster this difference is quickly reconciled via BlockUnderConstructionFeature#addReplicaIfNotPresent.

        Show
        arpitagarwal Arpit Agarwal added a comment - Thanks for the visualization Elek, Marton . This makes sense and the change looks good. Let's rename DatanodeStorageInfo#setRemaining to setRemainingForTests and make it package-private. A minor concern is the NameNode's mapping of (storage -> block replicas) can be temporarily incorrect. In a healthy cluster this difference is quickly reconciled via BlockUnderConstructionFeature#addReplicaIfNotPresent .
        Hide
        elek Elek, Marton added a comment -

        According to the suggestion of Arpit Agarwal I created a visual explanation (see the attached image) about this modification.

        This is the exact situation what the patched unit test uses:

        Without the patch the node2 would be used as the summary of the free spaces is the lowest there (2 GB). But there is a storage (storage5) where the free space is only 0.5 gb. So after the patch the storage 5 will be used: even if the overall free space on node4 is 100.5 Gb the storage5 has only 0.5 gb so it should be preferred to delete unnecessary blocks from there.

        Show
        elek Elek, Marton added a comment - According to the suggestion of Arpit Agarwal I created a visual explanation (see the attached image) about this modification. This is the exact situation what the patched unit test uses: Without the patch the node2 would be used as the summary of the free spaces is the lowest there (2 GB). But there is a storage (storage5) where the free space is only 0.5 gb. So after the patch the storage 5 will be used: even if the overall free space on node4 is 100.5 Gb the storage5 has only 0.5 gb so it should be preferred to delete unnecessary blocks from there.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 27s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 13m 30s trunk passed
        +1 compile 0m 46s trunk passed
        +1 checkstyle 0m 29s trunk passed
        +1 mvnsite 0m 51s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 46s trunk passed
        +1 javadoc 0m 40s trunk passed
        +1 mvninstall 0m 46s the patch passed
        +1 compile 0m 42s the patch passed
        +1 javac 0m 42s the patch passed
        +1 checkstyle 0m 25s the patch passed
        +1 mvnsite 0m 48s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 48s the patch passed
        +1 javadoc 0m 37s the patch passed
        -1 unit 102m 47s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 28s The patch does not generate ASF License warnings.
        128m 31s



        Reason Tests
        Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure060



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11295
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12849864/HDFS-11295.003.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux fd688a1462b3 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 312b36d
        Default Java 1.8.0_121
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18288/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18288/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18288/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 27s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 13m 30s trunk passed +1 compile 0m 46s trunk passed +1 checkstyle 0m 29s trunk passed +1 mvnsite 0m 51s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 46s trunk passed +1 javadoc 0m 40s trunk passed +1 mvninstall 0m 46s the patch passed +1 compile 0m 42s the patch passed +1 javac 0m 42s the patch passed +1 checkstyle 0m 25s the patch passed +1 mvnsite 0m 48s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 48s the patch passed +1 javadoc 0m 37s the patch passed -1 unit 102m 47s hadoop-hdfs in the patch failed. +1 asflicense 0m 28s The patch does not generate ASF License warnings. 128m 31s Reason Tests Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure060 Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11295 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12849864/HDFS-11295.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux fd688a1462b3 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 312b36d Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18288/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18288/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18288/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        elek Elek, Marton added a comment -

        Re uploading to trigger a new build.

        Show
        elek Elek, Marton added a comment - Re uploading to trigger a new build.
        Hide
        elek Elek, Marton added a comment -

        I am not sure what was the problem:

        Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.671 sec - in org.apache.hadoop.fs.TestFcHdfsCreateMkdir
        Tests run: 61, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.081 sec - in org.apache.hadoop.fs.TestSWebHdfsFileContextMainOperations
        Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.439 sec - in org.apache.hadoop.fs.TestUnbuffer
        Tests run: 10, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 24.75 sec - in org.apache.hadoop.fs.TestEnhancedByteBufferAccess
        
        Results :
        
        Tests run: 4967, Failures: 0, Errors: 0, Skipped: 48
        
        [INFO] ------------------------------------------------------------------------
        [INFO] BUILD FAILURE
        [INFO] ------------------------------------------------------------------------
        [INFO] Total time: 1:32:00.004s
        [INFO] Finished at: Sun Jan 22 22:26:18 UTC 2017
        [INFO] Final Memory: 29M/243M
        [INFO] ------------------------------------------------------------------------
        [WARNING] The requested profile "native" could not be activated because it does not exist.
        [WARNING] The requested profile "yarn-ui" could not be activated because it does not exist.
        [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on project hadoop-hdfs: There was a timeout or other error in the fork -> [Help 1]
        [ERROR] 
        [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
        [ERROR] Re-run Maven using the -X switch to enable full debug logging.
        

        I will upload the same patch again to trigger a new jenkins build.

        Show
        elek Elek, Marton added a comment - I am not sure what was the problem: Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.671 sec - in org.apache.hadoop.fs.TestFcHdfsCreateMkdir Tests run: 61, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.081 sec - in org.apache.hadoop.fs.TestSWebHdfsFileContextMainOperations Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.439 sec - in org.apache.hadoop.fs.TestUnbuffer Tests run: 10, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 24.75 sec - in org.apache.hadoop.fs.TestEnhancedByteBufferAccess Results : Tests run: 4967, Failures: 0, Errors: 0, Skipped: 48 [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 1:32:00.004s [INFO] Finished at: Sun Jan 22 22:26:18 UTC 2017 [INFO] Final Memory: 29M/243M [INFO] ------------------------------------------------------------------------ [WARNING] The requested profile " native " could not be activated because it does not exist. [WARNING] The requested profile "yarn-ui" could not be activated because it does not exist. [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test ( default -test) on project hadoop-hdfs: There was a timeout or other error in the fork -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch . [ERROR] Re-run Maven using the -X switch to enable full debug logging. I will upload the same patch again to trigger a new jenkins build.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 20s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 13m 36s trunk passed
        +1 compile 0m 46s trunk passed
        +1 checkstyle 0m 28s trunk passed
        +1 mvnsite 0m 52s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 44s trunk passed
        +1 javadoc 0m 39s trunk passed
        +1 mvninstall 0m 49s the patch passed
        +1 compile 0m 45s the patch passed
        +1 javac 0m 45s the patch passed
        +1 checkstyle 0m 25s the patch passed
        +1 mvnsite 0m 50s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 50s the patch passed
        +1 javadoc 0m 37s the patch passed
        -1 unit 92m 5s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 22s The patch does not generate ASF License warnings.
        117m 51s



        Reason Tests
        Timed out junit tests org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11295
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12848791/HDFS-11295.002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux e5580feeefde 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / a847903
        Default Java 1.8.0_121
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18239/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18239/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18239/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 13m 36s trunk passed +1 compile 0m 46s trunk passed +1 checkstyle 0m 28s trunk passed +1 mvnsite 0m 52s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 44s trunk passed +1 javadoc 0m 39s trunk passed +1 mvninstall 0m 49s the patch passed +1 compile 0m 45s the patch passed +1 javac 0m 45s the patch passed +1 checkstyle 0m 25s the patch passed +1 mvnsite 0m 50s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 50s the patch passed +1 javadoc 0m 37s the patch passed -1 unit 92m 5s hadoop-hdfs in the patch failed. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 117m 51s Reason Tests Timed out junit tests org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11295 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12848791/HDFS-11295.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e5580feeefde 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a847903 Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18239/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18239/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18239/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 22s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 14m 39s trunk passed
        +1 compile 0m 46s trunk passed
        +1 checkstyle 0m 26s trunk passed
        +1 mvnsite 0m 51s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 41s trunk passed
        +1 javadoc 0m 40s trunk passed
        +1 mvninstall 0m 46s the patch passed
        +1 compile 0m 42s the patch passed
        +1 javac 0m 42s the patch passed
        +1 checkstyle 0m 24s the patch passed
        +1 mvnsite 0m 46s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 45s the patch passed
        +1 javadoc 0m 35s the patch passed
        -1 unit 87m 38s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 30s The patch does not generate ASF License warnings.
        114m 10s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithNodeGroup
          hadoop.fs.viewfs.TestViewFileSystemHdfs



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11295
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12848760/HDFS-11295.001.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 33696ec648b9 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / a847903
        Default Java 1.8.0_121
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18235/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18235/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18235/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 14m 39s trunk passed +1 compile 0m 46s trunk passed +1 checkstyle 0m 26s trunk passed +1 mvnsite 0m 51s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 41s trunk passed +1 javadoc 0m 40s trunk passed +1 mvninstall 0m 46s the patch passed +1 compile 0m 42s the patch passed +1 javac 0m 42s the patch passed +1 checkstyle 0m 24s the patch passed +1 mvnsite 0m 46s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 45s the patch passed +1 javadoc 0m 35s the patch passed -1 unit 87m 38s hadoop-hdfs in the patch failed. +1 asflicense 0m 30s The patch does not generate ASF License warnings. 114m 10s Reason Tests Failed junit tests hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithNodeGroup   hadoop.fs.viewfs.TestViewFileSystemHdfs Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11295 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12848760/HDFS-11295.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 33696ec648b9 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a847903 Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18235/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18235/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18235/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.

          People

          • Assignee:
            elek Elek, Marton
            Reporter:
            Xiao Liang Xiao Liang
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development