Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None

      Description

      In the current design of HDFS Balancer, it doesn't use BlockPlacementPolicy used by namenode runtime. Instead, it has somewhat redundant code to make sure block allocation conforms with the rack policy.

      When namenode uses upgrade domain based policy, we need to make sure that HDFS balancer doesn't move blocks in a way that could violate upgrade domain block placement policy.

      In the longer term, we should consider how to make Balancer independent of the actual BlockPlacementPolicy as in HDFS-1431.

      1. HDFS-9007.patch
        51 kB
        Ming Ma
      2. HDFS-9007-2.patch
        52 kB
        Ming Ma
      3. HDFS-9007-branch-2.patch
        52 kB
        Ming Ma

        Activity

        Hide
        mingma Ming Ma added a comment -

        Here is the initial patch. The basic idea is to add a new method called isMovable to BlockPlacementPolicy for balancer and other tools to ask if the move is allowed.

        The reason we don't use verifyBlockPlacement in such scenario is isMovable and verifyBlockPlacement have different meanings. isMovable means "it is ok as long as it doesn't make things worse.". For example, say a block has two replicas and they are on the same rack. Moving one of the replicas to another node in the same rack is allowed from balancer's point of view. But verifyBlockPlacement won't allow that.

        The policy defined by isMovable is used by client-side tools. It is the same as the policy used by BlockManager to decide delete hint. Thus there is some refactoring to make sure that policy is shared properly.

        The patch also adds bunch of new tests and clean up some code along the way.

        Show
        mingma Ming Ma added a comment - Here is the initial patch. The basic idea is to add a new method called isMovable to BlockPlacementPolicy for balancer and other tools to ask if the move is allowed. The reason we don't use verifyBlockPlacement in such scenario is isMovable and verifyBlockPlacement have different meanings. isMovable means "it is ok as long as it doesn't make things worse.". For example, say a block has two replicas and they are on the same rack. Moving one of the replicas to another node in the same rack is allowed from balancer's point of view. But verifyBlockPlacement won't allow that. The policy defined by isMovable is used by client-side tools. It is the same as the policy used by BlockManager to decide delete hint. Thus there is some refactoring to make sure that policy is shared properly. The patch also adds bunch of new tests and clean up some code along the way.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        -1 pre-patch 18m 25s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 7 new or modified test files.
        +1 javac 8m 6s There were no new javac warning messages.
        +1 javadoc 10m 37s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        -1 checkstyle 1m 28s The applied patch generated 3 new checkstyle issues (total was 108, now 103).
        -1 whitespace 0m 9s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix.
        +1 install 1m 30s mvn install still works.
        +1 eclipse:eclipse 0m 36s The patch built with eclipse:eclipse.
        +1 findbugs 2m 33s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 native 3m 14s Pre-build of native portion
        -1 hdfs tests 80m 8s Tests failed in hadoop-hdfs.
            127m 12s  



        Reason Tests
        Failed unit tests hadoop.hdfs.server.blockmanagement.TestNodeCount
          hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
        Timed out tests org.apache.hadoop.hdfs.util.TestByteArrayManager



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12768831/HDFS-9007.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 56e4f62
        Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13207/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13207/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/13207/artifact/patchprocess/whitespace.txt
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13207/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13207/testReport/
        Java 1.7.0_55
        uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13207/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 18m 25s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 7 new or modified test files. +1 javac 8m 6s There were no new javac warning messages. +1 javadoc 10m 37s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 28s The applied patch generated 3 new checkstyle issues (total was 108, now 103). -1 whitespace 0m 9s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 30s mvn install still works. +1 eclipse:eclipse 0m 36s The patch built with eclipse:eclipse. +1 findbugs 2m 33s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 14s Pre-build of native portion -1 hdfs tests 80m 8s Tests failed in hadoop-hdfs.     127m 12s   Reason Tests Failed unit tests hadoop.hdfs.server.blockmanagement.TestNodeCount   hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot Timed out tests org.apache.hadoop.hdfs.util.TestByteArrayManager Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12768831/HDFS-9007.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 56e4f62 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13207/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13207/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/13207/artifact/patchprocess/whitespace.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13207/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13207/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13207/console This message was automatically generated.
        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        Hi, Ming Ma . This patch looks great to me.

        protected <T> DatanodeInfo getDatanodeInfo(T datanode) {
        

        I feel like it'd better to use function overloading here. It'd be safer because the compiler will handle the null case.

        +1 once address the comment.

        Show
        eddyxu Lei (Eddy) Xu added a comment - Hi, Ming Ma . This patch looks great to me. protected <T> DatanodeInfo getDatanodeInfo(T datanode) { I feel like it'd better to use function overloading here. It'd be safer because the compiler will handle the null case. +1 once address the comment.
        Hide
        mingma Ming Ma added a comment -

        Thanks Lei (Eddy) Xu! With function overloading of getDatanodeInfo, the generics functions can't compile as it doesn't know which getDatanodeInfo to use. Per offline discussion, Eddy suggested adding preconditions instead. Here is the updated patch.

        Show
        mingma Ming Ma added a comment - Thanks Lei (Eddy) Xu ! With function overloading of getDatanodeInfo, the generics functions can't compile as it doesn't know which getDatanodeInfo to use. Per offline discussion, Eddy suggested adding preconditions instead. Here is the updated patch.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 5s docker + precommit patch detected.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.
        +1 mvninstall 3m 7s trunk passed
        +1 compile 0m 31s trunk passed with JDK v1.8.0_60
        +1 compile 0m 30s trunk passed with JDK v1.7.0_79
        +1 checkstyle 0m 15s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        -1 findbugs 1m 50s hadoop-hdfs-project/hadoop-hdfs in trunk cannot run convertXmlToText from findbugs
        +1 javadoc 1m 7s trunk passed with JDK v1.8.0_60
        +1 javadoc 1m 46s trunk passed with JDK v1.7.0_79
        +1 mvninstall 0m 38s the patch passed
        +1 compile 0m 30s the patch passed with JDK v1.8.0_60
        +1 javac 0m 30s the patch passed
        +1 compile 0m 31s the patch passed with JDK v1.7.0_79
        +1 javac 0m 31s the patch passed
        -1 checkstyle 0m 15s Patch generated 3 new checkstyle issues in hadoop-hdfs-project/hadoop-hdfs (total was 109, now 104).
        +1 mvneclipse 0m 13s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 2s the patch passed
        +1 javadoc 1m 6s the patch passed with JDK v1.8.0_60
        +1 javadoc 1m 46s the patch passed with JDK v1.7.0_79
        -1 unit 51m 52s hadoop-hdfs in the patch failed with JDK v1.8.0_60.
        -1 unit 49m 53s hadoop-hdfs in the patch failed with JDK v1.7.0_79.
        -1 asflicense 0m 20s Patch generated 56 ASF License warnings.
        121m 1s



        Reason Tests
        JDK v1.8.0_60 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestXAttrWithSnapshot
          hadoop.hdfs.TestDFSUpgradeFromImage
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070
        JDK v1.7.0_79 Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160



        Subsystem Report/Notes
        Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-03
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12770257/HDFS-9007-2.patch
        JIRA Issue HDFS-9007
        Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile
        uname Linux efb30c1dc716 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-1a9afee/precommit/personality/hadoop.sh
        git revision trunk / 957f031
        Default Java 1.7.0_79
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
        findbugs v3.0.0
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/13354/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs.html
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13354/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/13354/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/13354/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt
        unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13354/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13354/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt
        JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13354/testReport/
        asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13354/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Max memory used 226MB
        Powered by Apache Yetus http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13354/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 5s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 7 new or modified test files. +1 mvninstall 3m 7s trunk passed +1 compile 0m 31s trunk passed with JDK v1.8.0_60 +1 compile 0m 30s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 15s trunk passed +1 mvneclipse 0m 14s trunk passed -1 findbugs 1m 50s hadoop-hdfs-project/hadoop-hdfs in trunk cannot run convertXmlToText from findbugs +1 javadoc 1m 7s trunk passed with JDK v1.8.0_60 +1 javadoc 1m 46s trunk passed with JDK v1.7.0_79 +1 mvninstall 0m 38s the patch passed +1 compile 0m 30s the patch passed with JDK v1.8.0_60 +1 javac 0m 30s the patch passed +1 compile 0m 31s the patch passed with JDK v1.7.0_79 +1 javac 0m 31s the patch passed -1 checkstyle 0m 15s Patch generated 3 new checkstyle issues in hadoop-hdfs-project/hadoop-hdfs (total was 109, now 104). +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 2s the patch passed +1 javadoc 1m 6s the patch passed with JDK v1.8.0_60 +1 javadoc 1m 46s the patch passed with JDK v1.7.0_79 -1 unit 51m 52s hadoop-hdfs in the patch failed with JDK v1.8.0_60. -1 unit 49m 53s hadoop-hdfs in the patch failed with JDK v1.7.0_79. -1 asflicense 0m 20s Patch generated 56 ASF License warnings. 121m 1s Reason Tests JDK v1.8.0_60 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestXAttrWithSnapshot   hadoop.hdfs.TestDFSUpgradeFromImage   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure070 JDK v1.7.0_79 Failed junit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160 Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-03 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12770257/HDFS-9007-2.patch JIRA Issue HDFS-9007 Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile uname Linux efb30c1dc716 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-1a9afee/precommit/personality/hadoop.sh git revision trunk / 957f031 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/13354/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13354/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13354/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13354/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13354/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13354/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13354/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13354/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 226MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13354/console This message was automatically generated.
        Hide
        mingma Ming Ma added a comment -

        Here is the patch for branch-2.

        Show
        mingma Ming Ma added a comment - Here is the patch for branch-2.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 10s docker + precommit patch detected.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.
        +1 mvninstall 8m 26s branch-2 passed
        +1 compile 0m 36s branch-2 passed with JDK v1.8.0_66
        +1 compile 0m 34s branch-2 passed with JDK v1.7.0_79
        +1 checkstyle 0m 17s branch-2 passed
        +1 mvneclipse 0m 17s branch-2 passed
        -1 findbugs 2m 9s hadoop-hdfs-project/hadoop-hdfs in branch-2 cannot run convertXmlToText from findbugs
        +1 javadoc 1m 11s branch-2 passed with JDK v1.8.0_66
        +1 javadoc 1m 55s branch-2 passed with JDK v1.7.0_79
        +1 mvninstall 0m 40s the patch passed
        +1 compile 0m 35s the patch passed with JDK v1.8.0_66
        +1 javac 0m 35s the patch passed
        +1 compile 0m 35s the patch passed with JDK v1.7.0_79
        +1 javac 0m 35s the patch passed
        -1 checkstyle 0m 15s Patch generated 3 new checkstyle issues in hadoop-hdfs-project/hadoop-hdfs (total was 109, now 104).
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 12s the patch passed
        +1 javadoc 1m 12s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 59s the patch passed with JDK v1.7.0_79
        -1 unit 63m 53s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
        -1 unit 66m 36s hadoop-hdfs in the patch failed with JDK v1.7.0_79.
        -1 asflicense 0m 22s Patch generated 56 ASF License warnings.
        157m 5s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.datanode.TestBlockScanner
          hadoop.hdfs.server.mover.TestStorageMover
          hadoop.hdfs.TestDistributedFileSystem
          hadoop.hdfs.TestRollingUpgradeRollback
          hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
        JDK v1.8.0_66 Timed out junit tests org.apache.hadoop.hdfs.TestAclsEndToEnd
        JDK v1.7.0_79 Failed junit tests hadoop.hdfs.TestDatanodeDeath
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
          hadoop.hdfs.TestDistributedFileSystem
          hadoop.hdfs.server.datanode.TestBlockReplacement
          hadoop.hdfs.TestRollingUpgradeRollback
          hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot



        Subsystem Report/Notes
        Docker Client=1.7.0 Server=1.7.0 Image:test-patch-base-hadoop-date2015-11-03
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12770393/HDFS-9007-branch-2.patch
        JIRA Issue HDFS-9007
        Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile
        uname Linux c5423100665d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-1a9afee/precommit/personality/hadoop.sh
        git revision branch-2 / 246ab44
        Default Java 1.7.0_79
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
        findbugs v3.0.0
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/13364/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs.html
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13364/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/13364/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/13364/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt
        unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13364/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13364/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt
        JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13364/testReport/
        asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13364/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Max memory used 213MB
        Powered by Apache Yetus http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13364/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 7 new or modified test files. +1 mvninstall 8m 26s branch-2 passed +1 compile 0m 36s branch-2 passed with JDK v1.8.0_66 +1 compile 0m 34s branch-2 passed with JDK v1.7.0_79 +1 checkstyle 0m 17s branch-2 passed +1 mvneclipse 0m 17s branch-2 passed -1 findbugs 2m 9s hadoop-hdfs-project/hadoop-hdfs in branch-2 cannot run convertXmlToText from findbugs +1 javadoc 1m 11s branch-2 passed with JDK v1.8.0_66 +1 javadoc 1m 55s branch-2 passed with JDK v1.7.0_79 +1 mvninstall 0m 40s the patch passed +1 compile 0m 35s the patch passed with JDK v1.8.0_66 +1 javac 0m 35s the patch passed +1 compile 0m 35s the patch passed with JDK v1.7.0_79 +1 javac 0m 35s the patch passed -1 checkstyle 0m 15s Patch generated 3 new checkstyle issues in hadoop-hdfs-project/hadoop-hdfs (total was 109, now 104). +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 12s the patch passed +1 javadoc 1m 12s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 59s the patch passed with JDK v1.7.0_79 -1 unit 63m 53s hadoop-hdfs in the patch failed with JDK v1.8.0_66. -1 unit 66m 36s hadoop-hdfs in the patch failed with JDK v1.7.0_79. -1 asflicense 0m 22s Patch generated 56 ASF License warnings. 157m 5s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.server.mover.TestStorageMover   hadoop.hdfs.TestDistributedFileSystem   hadoop.hdfs.TestRollingUpgradeRollback   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery JDK v1.8.0_66 Timed out junit tests org.apache.hadoop.hdfs.TestAclsEndToEnd JDK v1.7.0_79 Failed junit tests hadoop.hdfs.TestDatanodeDeath   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.TestDistributedFileSystem   hadoop.hdfs.server.datanode.TestBlockReplacement   hadoop.hdfs.TestRollingUpgradeRollback   hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot Subsystem Report/Notes Docker Client=1.7.0 Server=1.7.0 Image:test-patch-base-hadoop-date2015-11-03 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12770393/HDFS-9007-branch-2.patch JIRA Issue HDFS-9007 Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile uname Linux c5423100665d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-1a9afee/precommit/personality/hadoop.sh git revision branch-2 / 246ab44 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/13364/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13364/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13364/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13364/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13364/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13364/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13364/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13364/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 213MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13364/console This message was automatically generated.
        Hide
        mingma Ming Ma added a comment -

        The latest test is for the branch-2 patch. I ran the failed tests locally. They all passed except TestRollingUpgradeRollback which fails without the patch.

        Show
        mingma Ming Ma added a comment - The latest test is for the branch-2 patch. I ran the failed tests locally. They all passed except TestRollingUpgradeRollback which fails without the patch.
        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        Thanks much for this work, Ming Ma.

        +1. Committed.

        Show
        eddyxu Lei (Eddy) Xu added a comment - Thanks much for this work, Ming Ma . +1. Committed.
        Hide
        mingma Ming Ma added a comment -

        Thanks Lei (Eddy) Xu!

        Show
        mingma Ming Ma added a comment - Thanks Lei (Eddy) Xu !
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #8757 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8757/)
        HDFS-9007. Fix HDFS Balancer to honor upgrade domain policy. (Ming Ma (lei: rev ec414600ede8e305c584818565b50e055ea5d2b5)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithUpgradeDomain.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDNFencing.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithUpgradeDomain.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicy.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8757 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8757/ ) HDFS-9007 . Fix HDFS Balancer to honor upgrade domain policy. (Ming Ma (lei: rev ec414600ede8e305c584818565b50e055ea5d2b5) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithUpgradeDomain.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDNFencing.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithUpgradeDomain.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicy.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #639 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/639/)
        HDFS-9007. Fix HDFS Balancer to honor upgrade domain policy. (Ming Ma (lei: rev ec414600ede8e305c584818565b50e055ea5d2b5)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicy.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDNFencing.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithUpgradeDomain.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithUpgradeDomain.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #639 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/639/ ) HDFS-9007 . Fix HDFS Balancer to honor upgrade domain policy. (Ming Ma (lei: rev ec414600ede8e305c584818565b50e055ea5d2b5) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicy.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDNFencing.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithUpgradeDomain.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithUpgradeDomain.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2569 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2569/)
        HDFS-9007. Fix HDFS Balancer to honor upgrade domain policy. (Ming Ma (lei: rev ec414600ede8e305c584818565b50e055ea5d2b5)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithUpgradeDomain.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDNFencing.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicy.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithUpgradeDomain.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2569 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2569/ ) HDFS-9007 . Fix HDFS Balancer to honor upgrade domain policy. (Ming Ma (lei: rev ec414600ede8e305c584818565b50e055ea5d2b5) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithUpgradeDomain.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDNFencing.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicy.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithUpgradeDomain.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk #1362 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1362/)
        HDFS-9007. Fix HDFS Balancer to honor upgrade domain policy. (Ming Ma (lei: rev ec414600ede8e305c584818565b50e055ea5d2b5)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicy.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithUpgradeDomain.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDNFencing.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithUpgradeDomain.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #1362 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1362/ ) HDFS-9007 . Fix HDFS Balancer to honor upgrade domain policy. (Ming Ma (lei: rev ec414600ede8e305c584818565b50e055ea5d2b5) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicy.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithUpgradeDomain.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDNFencing.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithUpgradeDomain.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #628 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/628/)
        HDFS-9007. Fix HDFS Balancer to honor upgrade domain policy. (Ming Ma (lei: rev ec414600ede8e305c584818565b50e055ea5d2b5)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithUpgradeDomain.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicy.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithUpgradeDomain.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDNFencing.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #628 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/628/ ) HDFS-9007 . Fix HDFS Balancer to honor upgrade domain policy. (Ming Ma (lei: rev ec414600ede8e305c584818565b50e055ea5d2b5) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithUpgradeDomain.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicy.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithUpgradeDomain.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDNFencing.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #572 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/572/)
        HDFS-9007. Fix HDFS Balancer to honor upgrade domain policy. (Ming Ma (lei: rev ec414600ede8e305c584818565b50e055ea5d2b5)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithUpgradeDomain.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicy.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDNFencing.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithUpgradeDomain.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #572 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/572/ ) HDFS-9007 . Fix HDFS Balancer to honor upgrade domain policy. (Ming Ma (lei: rev ec414600ede8e305c584818565b50e055ea5d2b5) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithUpgradeDomain.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicy.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDNFencing.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithUpgradeDomain.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Hdfs-trunk #2510 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2510/)
        HDFS-9007. Fix HDFS Balancer to honor upgrade domain policy. (Ming Ma (lei: rev ec414600ede8e305c584818565b50e055ea5d2b5)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithUpgradeDomain.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDNFencing.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicy.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithUpgradeDomain.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #2510 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2510/ ) HDFS-9007 . Fix HDFS Balancer to honor upgrade domain policy. (Ming Ma (lei: rev ec414600ede8e305c584818565b50e055ea5d2b5) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithUpgradeDomain.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDNFencing.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicy.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithUpgradeDomain.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java

          People

          • Assignee:
            mingma Ming Ma
            Reporter:
            mingma Ming Ma
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development