Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: None
    • Labels:
      None

      Description

      When a node is transitioned to/stay in/transitioned out of maintenance state, we need to make sure blocks w.r.t. that nodes are properly handled.

      • When nodes are put into maintenance, it will first go to ENTERING_MAINTENANCE, and make sure blocks are minimally replicated before the nodes are transitioned to IN_MAINTENANCE.
      • Do not replica blocks when nodes are in maintenance states. Maintenance replica will remain in BlockMaps and thus is still considered valid from block replication point of view. In other words, putting a node to “maintenance” mode won’t trigger BlockManager to replicate its blocks.
      • Do not invalidate replicas on node under maintenance. After any file's replication factor is reduced, NN needs to invalidate some replicas. It should exclude nodes under maintenance in the handling.
      • Do not put IN_MAINTENANCE replicas in LocatedBlock for read operation.
      • Do not allocate any new block on nodes under maintenance.
      • Have Balancer exclude nodes under maintenance.
      • Exclude nodes under maintenance for DN cache.
      1. HDFS-9390.patch
        121 kB
        Ming Ma
      2. HDFS-9390-2.patch
        123 kB
        Ming Ma
      3. HDFS-9390-3.patch
        131 kB
        Ming Ma
      4. HDFS-9390-4.patch
        132 kB
        Ming Ma
      5. HDFS-9390-5.patch
        133 kB
        Ming Ma
      6. HDFS-9390-branch-2.002.patch
        124 kB
        Ming Ma
      7. HDFS-9390-branch-2.patch
        124 kB
        Ming Ma

        Activity

        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        Ping Ming Ma Is there any update on this issue?

        Show
        eddyxu Lei (Eddy) Xu added a comment - Ping Ming Ma Is there any update on this issue?
        Hide
        mingma Ming Ma added a comment -

        Lei (Eddy) Xu sorry for the delay. Due to the big difference between trunk and 2.6 which the initial patch is based on, it requires quite amount of work. Here is the draft patch. Couple notes:

        • Erasure coding might need more work, at least new unit tests are required. We can use another jira for that.
        • It seems the safety properties maintained by BlockManager is implied in the code. I have started to document more as part of this patch.
        • There are other issues the patch try to fix along the way, for example {BlockManager#getRedundancy}

          can be removed.

        Show
        mingma Ming Ma added a comment - Lei (Eddy) Xu sorry for the delay. Due to the big difference between trunk and 2.6 which the initial patch is based on, it requires quite amount of work. Here is the draft patch. Couple notes: Erasure coding might need more work, at least new unit tests are required. We can use another jira for that. It seems the safety properties maintained by BlockManager is implied in the code. I have started to document more as part of this patch. There are other issues the patch try to fix along the way, for example {BlockManager#getRedundancy} can be removed.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 16s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
        +1 mvninstall 8m 15s trunk passed
        +1 compile 0m 52s trunk passed
        +1 checkstyle 0m 42s trunk passed
        +1 mvnsite 0m 59s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 1m 53s trunk passed
        +1 javadoc 0m 59s trunk passed
        +1 mvninstall 0m 57s the patch passed
        +1 compile 0m 50s the patch passed
        +1 javac 0m 50s the patch passed
        -0 checkstyle 0m 40s hadoop-hdfs-project/hadoop-hdfs: The patch generated 18 new + 1059 unchanged - 7 fixed = 1077 total (was 1066)
        +1 mvnsite 1m 0s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 2s The patch has no ill-formed XML file.
        +1 findbugs 2m 3s the patch passed
        +1 javadoc 1m 0s the patch passed
        -1 unit 59m 5s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        81m 36s



        Reason Tests
        Failed junit tests hadoop.hdfs.TestMaintenanceState



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-9390
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830131/HDFS-9390.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux cd79f38546fb 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 6eb700e
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16851/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16851/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16851/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16851/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files. +1 mvninstall 8m 15s trunk passed +1 compile 0m 52s trunk passed +1 checkstyle 0m 42s trunk passed +1 mvnsite 0m 59s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 53s trunk passed +1 javadoc 0m 59s trunk passed +1 mvninstall 0m 57s the patch passed +1 compile 0m 50s the patch passed +1 javac 0m 50s the patch passed -0 checkstyle 0m 40s hadoop-hdfs-project/hadoop-hdfs: The patch generated 18 new + 1059 unchanged - 7 fixed = 1077 total (was 1066) +1 mvnsite 1m 0s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 2m 3s the patch passed +1 javadoc 1m 0s the patch passed -1 unit 59m 5s hadoop-hdfs in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 81m 36s Reason Tests Failed junit tests hadoop.hdfs.TestMaintenanceState Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-9390 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830131/HDFS-9390.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux cd79f38546fb 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6eb700e Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16851/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16851/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16851/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16851/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        mingma Ming Ma added a comment -

        Updated patch to address the checkstyle and test code issues.

        Show
        mingma Ming Ma added a comment - Updated patch to address the checkstyle and test code issues.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 19s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
        +1 mvninstall 8m 22s trunk passed
        +1 compile 0m 53s trunk passed
        +1 checkstyle 0m 41s trunk passed
        +1 mvnsite 0m 58s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 1m 50s trunk passed
        +1 javadoc 1m 1s trunk passed
        +1 mvninstall 0m 53s the patch passed
        +1 compile 0m 52s the patch passed
        +1 javac 0m 52s the patch passed
        -0 checkstyle 0m 37s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 1057 unchanged - 9 fixed = 1058 total (was 1066)
        +1 mvnsite 0m 58s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 1s The patch has no ill-formed XML file.
        +1 findbugs 2m 4s the patch passed
        +1 javadoc 0m 57s the patch passed
        -1 unit 66m 7s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 19s The patch does not generate ASF License warnings.
        88m 44s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-9390
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830375/HDFS-9390-2.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux e1b11e4d348c 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 4815d02
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16866/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16866/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16866/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16866/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files. +1 mvninstall 8m 22s trunk passed +1 compile 0m 53s trunk passed +1 checkstyle 0m 41s trunk passed +1 mvnsite 0m 58s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 50s trunk passed +1 javadoc 1m 1s trunk passed +1 mvninstall 0m 53s the patch passed +1 compile 0m 52s the patch passed +1 javac 0m 52s the patch passed -0 checkstyle 0m 37s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 1057 unchanged - 9 fixed = 1058 total (was 1066) +1 mvnsite 0m 58s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 4s the patch passed +1 javadoc 0m 57s the patch passed -1 unit 66m 7s hadoop-hdfs in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 88m 44s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-9390 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830375/HDFS-9390-2.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux e1b11e4d348c 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 4815d02 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16866/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16866/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16866/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16866/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        Hi, Ming Ma Thanks so much for posting this patch.

        It looks good to me overall. Some small nits:

        In HeartbeatManager.java#heartbeatCheck():

        try {
         dm.removeDeadDatanode(dead, !dead.isMaintenance());
        }
        

        If we change it to the following code, we can undo most of the DatanodeManager.java changes, of which the motivation of these changes are not clear to me in the first sight.

        if (!dead.isMaintenance()) {
           dm.removeDeadDatanode(dead);
        }
        

        Can you elaborate a little bit more about the following code?

          } else if (blockManager.getMinReplicationToBeInMaintenance() == 0) {
              LOG.info("MinReplicationToBeInMaintenance is set to zero. " + node +
                  " is put in maintenance state" + " immediately.");
              node.setInMaintenance();
            } else {
              stats.subtract(node);
              node.startMaintenance();
              stats.add(node);
            }
        

        Why it does not re-calculate stats when minReplicationToBeInMaintanence == 0?

        In DecommissionManager#startmaintance()

         // hbManager.startDecommission will set dead node to decommissioned.
        

        Is the comment correct in the context?

        One related question is that, why startMaintenance() and stopMaintenance() are in DecommissionManager.

        In NumberReplicas.java, you might want consider rename int maintenance() to int maintenanceReplicas, so is liveEnteringMaintence().

        Thanks.

        Show
        eddyxu Lei (Eddy) Xu added a comment - Hi, Ming Ma Thanks so much for posting this patch. It looks good to me overall. Some small nits: In HeartbeatManager.java#heartbeatCheck() : try { dm.removeDeadDatanode(dead, !dead.isMaintenance()); } If we change it to the following code, we can undo most of the DatanodeManager.java changes, of which the motivation of these changes are not clear to me in the first sight. if (!dead.isMaintenance()) { dm.removeDeadDatanode(dead); } Can you elaborate a little bit more about the following code? } else if (blockManager.getMinReplicationToBeInMaintenance() == 0) { LOG.info( "MinReplicationToBeInMaintenance is set to zero. " + node + " is put in maintenance state" + " immediately." ); node.setInMaintenance(); } else { stats.subtract(node); node.startMaintenance(); stats.add(node); } Why it does not re-calculate stats when minReplicationToBeInMaintanence == 0 ? In DecommissionManager#startmaintance() // hbManager.startDecommission will set dead node to decommissioned. Is the comment correct in the context? One related question is that, why startMaintenance() and stopMaintenance() are in DecommissionManager . In NumberReplicas.java , you might want consider rename int maintenance() to int maintenanceReplicas , so is liveEnteringMaintence() . Thanks.
        Hide
        mingma Ming Ma added a comment -

        Thanks Lei (Eddy) Xu for the review! Here is the new patch.

        If we change it to the following code, we can undo most of the DatanodeManager.java changes, of which the motivation of these changes are not clear to me in the first sight.

        The main reason is DatanodeManager#removeDatanode performs other operations such as heartbeatManager.removeDatanode(nodeInfo); and blockManager.getBlockReportLeaseManager().unregister(nodeInfo); which should be called when a maintenance node becomes dead.

        Why it does not re-calculate stats when minReplicationToBeInMaintanence == 0?

        Good catch. In addition to fixing it, the new patch also updates TestNamenodeCapacityReport to cover maintenance scenario.

        Is the comment correct in the context?

        Fixed.

        One related question is that, why startMaintenance() and stopMaintenance() are in DecommissionManager.

        This is similar to startDecommission() and stopDecommission() in DecommissionManager. I plan to rename DecommissionManager to AdminServiceManager as part of HDFS-9388.

        In NumberReplicas.java, you might want consider rename int maintenance() to int maintenanceReplicas, so is liveEnteringMaintence().

        Fixed.

        Show
        mingma Ming Ma added a comment - Thanks Lei (Eddy) Xu for the review! Here is the new patch. If we change it to the following code, we can undo most of the DatanodeManager.java changes, of which the motivation of these changes are not clear to me in the first sight. The main reason is DatanodeManager#removeDatanode performs other operations such as heartbeatManager.removeDatanode(nodeInfo); and blockManager.getBlockReportLeaseManager().unregister(nodeInfo); which should be called when a maintenance node becomes dead. Why it does not re-calculate stats when minReplicationToBeInMaintanence == 0? Good catch. In addition to fixing it, the new patch also updates TestNamenodeCapacityReport to cover maintenance scenario. Is the comment correct in the context? Fixed. One related question is that, why startMaintenance() and stopMaintenance() are in DecommissionManager. This is similar to startDecommission() and stopDecommission() in DecommissionManager. I plan to rename DecommissionManager to AdminServiceManager as part of HDFS-9388 . In NumberReplicas.java, you might want consider rename int maintenance() to int maintenanceReplicas, so is liveEnteringMaintence(). Fixed.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 12s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.
        +1 mvninstall 7m 21s trunk passed
        +1 compile 0m 48s trunk passed
        +1 checkstyle 0m 43s trunk passed
        +1 mvnsite 0m 54s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 1m 42s trunk passed
        +1 javadoc 1m 0s trunk passed
        +1 mvninstall 0m 47s the patch passed
        +1 compile 0m 44s the patch passed
        +1 javac 0m 44s the patch passed
        -0 checkstyle 0m 36s hadoop-hdfs-project/hadoop-hdfs: The patch generated 5 new + 1067 unchanged - 15 fixed = 1072 total (was 1082)
        +1 mvnsite 0m 57s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 2s The patch has no ill-formed XML file.
        +1 findbugs 1m 56s the patch passed
        +1 javadoc 1m 0s the patch passed
        -1 unit 76m 44s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        97m 33s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-9390
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830933/HDFS-9390-3.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux fd4a48e3650c 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 236ac77
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16930/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16930/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16930/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16930/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 7 new or modified test files. +1 mvninstall 7m 21s trunk passed +1 compile 0m 48s trunk passed +1 checkstyle 0m 43s trunk passed +1 mvnsite 0m 54s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 42s trunk passed +1 javadoc 1m 0s trunk passed +1 mvninstall 0m 47s the patch passed +1 compile 0m 44s the patch passed +1 javac 0m 44s the patch passed -0 checkstyle 0m 36s hadoop-hdfs-project/hadoop-hdfs: The patch generated 5 new + 1067 unchanged - 15 fixed = 1072 total (was 1082) +1 mvnsite 0m 57s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 1m 56s the patch passed +1 javadoc 1m 0s the patch passed -1 unit 76m 44s hadoop-hdfs in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 97m 33s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-9390 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830933/HDFS-9390-3.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux fd4a48e3650c 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 236ac77 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16930/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16930/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16930/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16930/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        mingma Ming Ma added a comment -

        Updated patch to address checkstyle issues.

        Show
        mingma Ming Ma added a comment - Updated patch to address checkstyle issues.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.
        +1 mvninstall 7m 42s trunk passed
        +1 compile 0m 49s trunk passed
        +1 checkstyle 0m 40s trunk passed
        +1 mvnsite 0m 59s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 51s trunk passed
        +1 javadoc 0m 58s trunk passed
        +1 mvninstall 0m 51s the patch passed
        +1 compile 0m 48s the patch passed
        +1 javac 0m 48s the patch passed
        +1 checkstyle 0m 37s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 1067 unchanged - 15 fixed = 1067 total (was 1082)
        +1 mvnsite 0m 55s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 2s The patch has no ill-formed XML file.
        +1 findbugs 1m 57s the patch passed
        +1 javadoc 0m 56s the patch passed
        -1 unit 62m 11s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 23s The patch does not generate ASF License warnings.
        83m 42s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-9390
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12831013/HDFS-9390-4.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux f3510b670b7a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 10be459
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16935/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16935/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16935/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 7 new or modified test files. +1 mvninstall 7m 42s trunk passed +1 compile 0m 49s trunk passed +1 checkstyle 0m 40s trunk passed +1 mvnsite 0m 59s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 51s trunk passed +1 javadoc 0m 58s trunk passed +1 mvninstall 0m 51s the patch passed +1 compile 0m 48s the patch passed +1 javac 0m 48s the patch passed +1 checkstyle 0m 37s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 1067 unchanged - 15 fixed = 1067 total (was 1082) +1 mvnsite 0m 55s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 1m 57s the patch passed +1 javadoc 0m 56s the patch passed -1 unit 62m 11s hadoop-hdfs in the patch failed. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 83m 42s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-9390 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12831013/HDFS-9390-4.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux f3510b670b7a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 10be459 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16935/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16935/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16935/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        eddyxu Lei (Eddy) Xu added a comment -

        Hi, Ming Ma

        The latest patch needs to rebase to trunk. But the code LGTM. +1 pending the rebase.

        Thanks for the great effort!

        Show
        eddyxu Lei (Eddy) Xu added a comment - Hi, Ming Ma The latest patch needs to rebase to trunk. But the code LGTM. +1 pending the rebase. Thanks for the great effort!
        Hide
        mingma Ming Ma added a comment -

        Thanks Lei (Eddy) Xu! Here is the patch after rebase.

        Show
        mingma Ming Ma added a comment - Thanks Lei (Eddy) Xu ! Here is the patch after rebase.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        -1 patch 0m 5s HDFS-9390 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



        Subsystem Report/Notes
        JIRA Issue HDFS-9390
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833626/HDFS-9390-5.patch
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17173/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 5s HDFS-9390 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Issue HDFS-9390 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833626/HDFS-9390-5.patch Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17173/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        mingma Ming Ma added a comment -

        Reload the correct patch.

        Show
        mingma Ming Ma added a comment - Reload the correct patch.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 20s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.
        +1 mvninstall 6m 46s trunk passed
        +1 compile 0m 44s trunk passed
        +1 checkstyle 0m 40s trunk passed
        +1 mvnsite 0m 50s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 1m 41s trunk passed
        +1 javadoc 0m 38s trunk passed
        +1 mvninstall 0m 44s the patch passed
        +1 compile 0m 43s the patch passed
        +1 javac 0m 43s the patch passed
        +1 checkstyle 0m 35s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 1062 unchanged - 16 fixed = 1062 total (was 1078)
        +1 mvnsite 0m 50s the patch passed
        +1 mvneclipse 0m 9s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 1s The patch has no ill-formed XML file.
        +1 findbugs 1m 48s the patch passed
        +1 javadoc 0m 39s the patch passed
        -1 unit 76m 26s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        95m 27s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-9390
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833631/HDFS-9390-5.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux 0be78b44f76d 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 1f304b0
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/17174/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17174/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17174/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 7 new or modified test files. +1 mvninstall 6m 46s trunk passed +1 compile 0m 44s trunk passed +1 checkstyle 0m 40s trunk passed +1 mvnsite 0m 50s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 41s trunk passed +1 javadoc 0m 38s trunk passed +1 mvninstall 0m 44s the patch passed +1 compile 0m 43s the patch passed +1 javac 0m 43s the patch passed +1 checkstyle 0m 35s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 1062 unchanged - 16 fixed = 1062 total (was 1078) +1 mvnsite 0m 50s the patch passed +1 mvneclipse 0m 9s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 1m 48s the patch passed +1 javadoc 0m 39s the patch passed -1 unit 76m 26s hadoop-hdfs in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 95m 27s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-9390 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833631/HDFS-9390-5.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 0be78b44f76d 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 1f304b0 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/17174/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17174/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17174/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        mingma Ming Ma added a comment -

        Here is the patch for branch-2. Note that the backport isn't trivial due to difference between branch-2 and trunk.

        Show
        mingma Ming Ma added a comment - Here is the patch for branch-2. Note that the backport isn't trivial due to difference between branch-2 and trunk.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 21s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.
        -1 mvninstall 0m 45s root in branch-2 failed.
        +1 compile 0m 43s branch-2 passed with JDK v1.8.0_101
        +1 compile 0m 41s branch-2 passed with JDK v1.7.0_111
        +1 checkstyle 0m 38s branch-2 passed
        +1 mvnsite 0m 50s branch-2 passed
        +1 mvneclipse 0m 13s branch-2 passed
        +1 findbugs 1m 54s branch-2 passed
        +1 javadoc 0m 51s branch-2 passed with JDK v1.8.0_101
        +1 javadoc 1m 34s branch-2 passed with JDK v1.7.0_111
        -1 mvninstall 0m 43s hadoop-hdfs in the patch failed.
        +1 compile 0m 36s the patch passed with JDK v1.8.0_101
        +1 javac 0m 36s the patch passed
        +1 compile 0m 40s the patch passed with JDK v1.7.0_111
        +1 javac 0m 40s the patch passed
        -0 checkstyle 0m 38s hadoop-hdfs-project/hadoop-hdfs: The patch generated 6 new + 1051 unchanged - 21 fixed = 1057 total (was 1072)
        +1 mvnsite 0m 49s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 1s The patch has no ill-formed XML file.
        +1 findbugs 2m 7s the patch passed
        +1 javadoc 0m 52s the patch passed with JDK v1.8.0_101
        +1 javadoc 1m 35s the patch passed with JDK v1.7.0_111
        +1 unit 64m 24s hadoop-hdfs in the patch passed with JDK v1.7.0_111.
        +1 asflicense 0m 22s The patch does not generate ASF License warnings.
        143m 48s



        Reason Tests
        JDK v1.8.0_101 Failed junit tests hadoop.hdfs.server.namenode.TestNamenodeCapacityReport



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:b59b8b7
        JIRA Issue HDFS-9390
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833645/HDFS-9390-branch-2.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux 7531c9a4e757 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision branch-2 / 439422f
        Default Java 1.7.0_111
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
        mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/17175/artifact/patchprocess/branch-mvninstall-root.txt
        findbugs v3.0.0
        mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/17175/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17175/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17175/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17175/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 7 new or modified test files. -1 mvninstall 0m 45s root in branch-2 failed. +1 compile 0m 43s branch-2 passed with JDK v1.8.0_101 +1 compile 0m 41s branch-2 passed with JDK v1.7.0_111 +1 checkstyle 0m 38s branch-2 passed +1 mvnsite 0m 50s branch-2 passed +1 mvneclipse 0m 13s branch-2 passed +1 findbugs 1m 54s branch-2 passed +1 javadoc 0m 51s branch-2 passed with JDK v1.8.0_101 +1 javadoc 1m 34s branch-2 passed with JDK v1.7.0_111 -1 mvninstall 0m 43s hadoop-hdfs in the patch failed. +1 compile 0m 36s the patch passed with JDK v1.8.0_101 +1 javac 0m 36s the patch passed +1 compile 0m 40s the patch passed with JDK v1.7.0_111 +1 javac 0m 40s the patch passed -0 checkstyle 0m 38s hadoop-hdfs-project/hadoop-hdfs: The patch generated 6 new + 1051 unchanged - 21 fixed = 1057 total (was 1072) +1 mvnsite 0m 49s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 7s the patch passed +1 javadoc 0m 52s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 35s the patch passed with JDK v1.7.0_111 +1 unit 64m 24s hadoop-hdfs in the patch passed with JDK v1.7.0_111. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 143m 48s Reason Tests JDK v1.8.0_101 Failed junit tests hadoop.hdfs.server.namenode.TestNamenodeCapacityReport Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HDFS-9390 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833645/HDFS-9390-branch-2.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 7531c9a4e757 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 439422f Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/17175/artifact/patchprocess/branch-mvninstall-root.txt findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/17175/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17175/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17175/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17175/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        mingma Ming Ma added a comment - - edited

        Updated branch-2 patch to address some of the checkstyle issues.

        Show
        mingma Ming Ma added a comment - - edited Updated branch-2 patch to address some of the checkstyle issues.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        -1 patch 0m 7s HDFS-9390 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



        Subsystem Report/Notes
        JIRA Issue HDFS-9390
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833649/HDFS-9390-2-branch-2.patch
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17176/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 patch 0m 7s HDFS-9390 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Issue HDFS-9390 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833649/HDFS-9390-2-branch-2.patch Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17176/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        mingma Ming Ma added a comment -

        Reload with the proper patch name for Jenkins to run.

        Show
        mingma Ming Ma added a comment - Reload with the proper patch name for Jenkins to run.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 21s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.
        -1 mvninstall 0m 44s root in branch-2 failed.
        +1 compile 0m 37s branch-2 passed with JDK v1.8.0_101
        +1 compile 0m 41s branch-2 passed with JDK v1.7.0_111
        +1 checkstyle 0m 36s branch-2 passed
        +1 mvnsite 0m 52s branch-2 passed
        +1 mvneclipse 0m 12s branch-2 passed
        +1 findbugs 1m 55s branch-2 passed
        +1 javadoc 0m 52s branch-2 passed with JDK v1.8.0_101
        +1 javadoc 1m 33s branch-2 passed with JDK v1.7.0_111
        -1 mvninstall 0m 43s hadoop-hdfs in the patch failed.
        +1 compile 0m 36s the patch passed with JDK v1.8.0_101
        +1 javac 0m 36s the patch passed
        +1 compile 0m 41s the patch passed with JDK v1.7.0_111
        +1 javac 0m 41s the patch passed
        -0 checkstyle 0m 38s hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 1051 unchanged - 21 fixed = 1055 total (was 1072)
        +1 mvnsite 0m 50s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 1s The patch has no ill-formed XML file.
        +1 findbugs 2m 8s the patch passed
        +1 javadoc 0m 52s the patch passed with JDK v1.8.0_101
        +1 javadoc 1m 33s the patch passed with JDK v1.7.0_111
        +1 unit 60m 58s hadoop-hdfs in the patch passed with JDK v1.7.0_111.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        138m 37s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:b59b8b7
        JIRA Issue HDFS-9390
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833661/HDFS-9390-branch-2.002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux 6b46ecb0f87e 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision branch-2 / 439422f
        Default Java 1.7.0_111
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
        mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/17177/artifact/patchprocess/branch-mvninstall-root.txt
        findbugs v3.0.0
        mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/17177/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17177/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17177/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17177/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 7 new or modified test files. -1 mvninstall 0m 44s root in branch-2 failed. +1 compile 0m 37s branch-2 passed with JDK v1.8.0_101 +1 compile 0m 41s branch-2 passed with JDK v1.7.0_111 +1 checkstyle 0m 36s branch-2 passed +1 mvnsite 0m 52s branch-2 passed +1 mvneclipse 0m 12s branch-2 passed +1 findbugs 1m 55s branch-2 passed +1 javadoc 0m 52s branch-2 passed with JDK v1.8.0_101 +1 javadoc 1m 33s branch-2 passed with JDK v1.7.0_111 -1 mvninstall 0m 43s hadoop-hdfs in the patch failed. +1 compile 0m 36s the patch passed with JDK v1.8.0_101 +1 javac 0m 36s the patch passed +1 compile 0m 41s the patch passed with JDK v1.7.0_111 +1 javac 0m 41s the patch passed -0 checkstyle 0m 38s hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 1051 unchanged - 21 fixed = 1055 total (was 1072) +1 mvnsite 0m 50s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 8s the patch passed +1 javadoc 0m 52s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 33s the patch passed with JDK v1.7.0_111 +1 unit 60m 58s hadoop-hdfs in the patch passed with JDK v1.7.0_111. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 138m 37s Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HDFS-9390 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833661/HDFS-9390-branch-2.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 6b46ecb0f87e 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 439422f Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/17177/artifact/patchprocess/branch-mvninstall-root.txt findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/17177/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17177/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17177/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17177/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        mingma Ming Ma added a comment -

        Thanks Lei (Eddy) Xu again. I have committed it to trunk and branch-2.

        Show
        mingma Ming Ma added a comment - Thanks Lei (Eddy) Xu again. I have committed it to trunk and branch-2.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10628 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10628/)
        HDFS-9390. Block management for maintenance states. (mingma: rev b61fb267b92b2736920b4bd0c673d31e7632ebb9)

        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/ErasureCodingWork.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMaintenanceState.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDecommissioningStatus.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/StorageTypeStats.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/LowRedundancyBlocks.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HeartbeatManager.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDecommission.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/CacheReplicationMonitor.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/util/HostsFileWriter.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/AdminStatesBaseTest.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNamenodeCapacityReport.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10628 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10628/ ) HDFS-9390 . Block management for maintenance states. (mingma: rev b61fb267b92b2736920b4bd0c673d31e7632ebb9) (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/ErasureCodingWork.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMaintenanceState.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDecommissioningStatus.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/StorageTypeStats.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/LowRedundancyBlocks.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HeartbeatManager.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDecommission.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/CacheReplicationMonitor.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/util/HostsFileWriter.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/AdminStatesBaseTest.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNamenodeCapacityReport.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 20s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 7 new or modified test files.
        +1 mvninstall 7m 46s branch-2 passed
        +1 compile 0m 49s branch-2 passed with JDK v1.8.0_101
        +1 compile 0m 48s branch-2 passed with JDK v1.7.0_111
        +1 checkstyle 0m 42s branch-2 passed
        +1 mvnsite 0m 55s branch-2 passed
        +1 mvneclipse 0m 17s branch-2 passed
        +1 findbugs 2m 7s branch-2 passed
        +1 javadoc 1m 1s branch-2 passed with JDK v1.8.0_101
        +1 javadoc 1m 46s branch-2 passed with JDK v1.7.0_111
        +1 mvninstall 0m 49s the patch passed
        +1 compile 0m 42s the patch passed with JDK v1.8.0_101
        +1 javac 0m 42s the patch passed
        +1 compile 0m 48s the patch passed with JDK v1.7.0_111
        +1 javac 0m 48s the patch passed
        -0 checkstyle 0m 40s hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 1050 unchanged - 21 fixed = 1054 total (was 1071)
        +1 mvnsite 0m 56s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 1s The patch has no ill-formed XML file.
        +1 findbugs 2m 19s the patch passed
        +1 javadoc 0m 59s the patch passed with JDK v1.8.0_101
        +1 javadoc 1m 42s the patch passed with JDK v1.7.0_111
        +1 unit 49m 23s hadoop-hdfs in the patch passed with JDK v1.7.0_111.
        +1 asflicense 0m 20s The patch does not generate ASF License warnings.
        132m 6s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:b59b8b7
        JIRA Issue HDFS-9390
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833661/HDFS-9390-branch-2.002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
        uname Linux b5e02a11eca3 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision branch-2 / e02c756
        Default Java 1.7.0_111
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17190/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17190/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17190/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 7 new or modified test files. +1 mvninstall 7m 46s branch-2 passed +1 compile 0m 49s branch-2 passed with JDK v1.8.0_101 +1 compile 0m 48s branch-2 passed with JDK v1.7.0_111 +1 checkstyle 0m 42s branch-2 passed +1 mvnsite 0m 55s branch-2 passed +1 mvneclipse 0m 17s branch-2 passed +1 findbugs 2m 7s branch-2 passed +1 javadoc 1m 1s branch-2 passed with JDK v1.8.0_101 +1 javadoc 1m 46s branch-2 passed with JDK v1.7.0_111 +1 mvninstall 0m 49s the patch passed +1 compile 0m 42s the patch passed with JDK v1.8.0_101 +1 javac 0m 42s the patch passed +1 compile 0m 48s the patch passed with JDK v1.7.0_111 +1 javac 0m 48s the patch passed -0 checkstyle 0m 40s hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 1050 unchanged - 21 fixed = 1054 total (was 1071) +1 mvnsite 0m 56s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 19s the patch passed +1 javadoc 0m 59s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 42s the patch passed with JDK v1.7.0_111 +1 unit 49m 23s hadoop-hdfs in the patch passed with JDK v1.7.0_111. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 132m 6s Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HDFS-9390 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833661/HDFS-9390-branch-2.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux b5e02a11eca3 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / e02c756 Default Java 1.7.0_111 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/17190/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt JDK v1.7.0_111 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17190/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17190/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.

          People

          • Assignee:
            mingma Ming Ma
            Reporter:
            mingma Ming Ma
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development