Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7933

fsck should also report decommissioning replicas.

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 2.7.4, 3.0.0-alpha1
    • Component/s: namenode
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      The output of hdfs fsck now also contains information about decommissioning replicas.

      Description

      Fsck doesn't count replicas that are on decommissioning nodes. If a block has all replicas on the decommissioning nodes, it will be marked as missing, which is alarming for the admins, although the system will replicate them before nodes are decommissioned.
      Fsck output should also show decommissioning replicas along with the live replicas.

      1. HDFS-7933.00.patch
        8 kB
        Xiaoyu Yao
      2. HDFS-7933.01.patch
        9 kB
        Xiaoyu Yao
      3. HDFS-7933.02.patch
        23 kB
        Xiaoyu Yao
      4. HDFS-7933.03.patch
        23 kB
        Xiaoyu Yao
      5. HDFS-7933-branch-2.7.00.patch
        23 kB
        Zhe Zhang

        Issue Links

          Activity

          Hide
          xyao Xiaoyu Yao added a comment -

          Attach a patch that removes the corruption warning for decommissioned replicas and shows them separately with live replicas.

          Show
          xyao Xiaoyu Yao added a comment - Attach a patch that removes the corruption warning for decommissioned replicas and shows them separately with live replicas.
          Hide
          xyao Xiaoyu Yao added a comment -

          Sample output from unit test during decommission:

          Status: HEALTHY
           Total size:	512 B
           Total dirs:	0
           Total files:	1
           Total symlinks:		0
           Total blocks (validated):	1 (avg. block size 512 B)
           Minimally replicated blocks:	1 (100.0 %)
           Over-replicated blocks:	0 (0.0 %)
           Under-replicated blocks:	0 (0.0 %)
           Mis-replicated blocks:		0 (0.0 %)
           Default replication factor:	1
           Average block replication:	2.0
           Corrupt blocks:		0
           Missing replicas:		0
           DecommissioningReplicas:		1 (0.0 %)
           Number of data-nodes:		2
           Number of racks:		2
          FSCK ended at Mon Mar 16 15:04:21 PDT 2015 in 1 milliseconds
          
          
          The filesystem under path '/testfile' is HEALTHY
          
          Show
          xyao Xiaoyu Yao added a comment - Sample output from unit test during decommission: Status: HEALTHY Total size: 512 B Total dirs: 0 Total files: 1 Total symlinks: 0 Total blocks (validated): 1 (avg. block size 512 B) Minimally replicated blocks: 1 (100.0 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks: 0 (0.0 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor: 1 Average block replication: 2.0 Corrupt blocks: 0 Missing replicas: 0 DecommissioningReplicas: 1 (0.0 %) Number of data-nodes: 2 Number of racks: 2 FSCK ended at Mon Mar 16 15:04:21 PDT 2015 in 1 milliseconds The filesystem under path '/testfile' is HEALTHY
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12704903/HDFS-7933.00.patch
          against trunk revision 2681ed9.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.TestClientReportBadBlock

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9905//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9905//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12704903/HDFS-7933.00.patch against trunk revision 2681ed9. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.TestClientReportBadBlock Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9905//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9905//console This message is automatically generated.
          Hide
          xyao Xiaoyu Yao added a comment -

          Fix the unit test failure from TestClientReportBadBlock due to fsck output string change.

          Show
          xyao Xiaoyu Yao added a comment - Fix the unit test failure from TestClientReportBadBlock due to fsck output string change.
          Hide
          jnp Jitendra Nath Pandey added a comment -

          Xiaoyu Yao, one of the key issues is that NumReplicas doesn't distinguish between decommissioned or decommissioning replicas, and is initialized with the sum of decommissioning and decommissioned replicas. I think fsck should report decommissioning and decommissioned replicas separately to provide the complete picture.

          Show
          jnp Jitendra Nath Pandey added a comment - Xiaoyu Yao , one of the key issues is that NumReplicas doesn't distinguish between decommissioned or decommissioning replicas, and is initialized with the sum of decommissioning and decommissioned replicas. I think fsck should report decommissioning and decommissioned replicas separately to provide the complete picture.
          Hide
          xyao Xiaoyu Yao added a comment -

          Thanks Jitendra Nath Pandey for reviewing the patch. I've updated the patch based on your feedback.

          Summary of changes:

          1) adding NumberReplicas#decommissioned and NumberReplicas#decommissioning to track the decommissioned and decommissioning replicas, respectively.

          2) deprecating NumberReplicas#decommissionedReplicas() by NumberReplicas#decommissionedAndDecommissioning() to avoid the misleading name.

          3) Display decommissioning and decommissioned replica separately in NamenodeFsck#check().

          Show
          xyao Xiaoyu Yao added a comment - Thanks Jitendra Nath Pandey for reviewing the patch. I've updated the patch based on your feedback. Summary of changes: 1) adding NumberReplicas#decommissioned and NumberReplicas#decommissioning to track the decommissioned and decommissioning replicas, respectively. 2) deprecating NumberReplicas#decommissionedReplicas() by NumberReplicas#decommissionedAndDecommissioning() to avoid the misleading name. 3) Display decommissioning and decommissioned replica separately in NamenodeFsck#check().
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12708365/HDFS-7933.02.patch
          against trunk revision 85dc3c1.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
          org.apache.hadoop.hdfs.server.namenode.TestDefaultBlockPlacementPolicy

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10126//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10126//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12708365/HDFS-7933.02.patch against trunk revision 85dc3c1. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA org.apache.hadoop.hdfs.server.namenode.TestDefaultBlockPlacementPolicy Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10126//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10126//console This message is automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          Hi Xiaoyu. This patch looks good. Here are a few comments:

          1. In BlockManager#chooseSourceDatanode, should the decommissioning counter be incremented by countableReplica, like for the other counters?
                  if ((nodesCorrupt != null) && (nodesCorrupt.contains(node)))
                    corrupt += countableReplica;
                  else if (node.isDecommissionInProgress()) {
                    decommissioning++;
                  } else if (node.isDecommissioned()) {
                    decommissioned += countableReplica;
            
          2. Could the deprecated NumberReplicas#decommissionedReplicas be implemented to forward the call to decommissionedAndDecommissioning? If so, then we can eliminate the decommissionedReplicas member variable.
          3. The new test in TestFsck appears to create a StringBuilder, append data to it, and then never use it for anything. Can it be removed?
          Show
          cnauroth Chris Nauroth added a comment - Hi Xiaoyu. This patch looks good. Here are a few comments: In BlockManager#chooseSourceDatanode , should the decommissioning counter be incremented by countableReplica , like for the other counters? if ((nodesCorrupt != null ) && (nodesCorrupt.contains(node))) corrupt += countableReplica; else if (node.isDecommissionInProgress()) { decommissioning++; } else if (node.isDecommissioned()) { decommissioned += countableReplica; Could the deprecated NumberReplicas#decommissionedReplicas be implemented to forward the call to decommissionedAndDecommissioning ? If so, then we can eliminate the decommissionedReplicas member variable. The new test in TestFsck appears to create a StringBuilder , append data to it, and then never use it for anything. Can it be removed?
          Hide
          xyao Xiaoyu Yao added a comment -

          Thanks Chris for the review. I've updated the patch based on your comments.

          In BlockManager#chooseSourceDatanode, should the decommissioning counter be incremented by countableReplica, like for the other counters?

          Good catch. Fixed.

          Could the deprecated NumberReplicas#decommissionedReplicas be implemented to forward the call to decommissionedAndDecommissioning? If so, then we can eliminate the decommissionedReplicas member variable.

          Agree and fixed.

          The new test in TestFsck appears to create a StringBuilder, append data to it, and then never use it for anything. Can it be removed?

          Fixed.

          Show
          xyao Xiaoyu Yao added a comment - Thanks Chris for the review. I've updated the patch based on your comments. In BlockManager#chooseSourceDatanode, should the decommissioning counter be incremented by countableReplica, like for the other counters? Good catch. Fixed. Could the deprecated NumberReplicas#decommissionedReplicas be implemented to forward the call to decommissionedAndDecommissioning? If so, then we can eliminate the decommissionedReplicas member variable. Agree and fixed. The new test in TestFsck appears to create a StringBuilder, append data to it, and then never use it for anything. Can it be removed? Fixed.
          Hide
          cnauroth Chris Nauroth added a comment -

          +1 for patch v03 pending Jenkins. Thanks for incorporating the feedback. I'm targeting this to 2.8.0 since the release process has begun for 2.7.0.

          Show
          cnauroth Chris Nauroth added a comment - +1 for patch v03 pending Jenkins. Thanks for incorporating the feedback. I'm targeting this to 2.8.0 since the release process has begun for 2.7.0.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12724284/HDFS-7933.03.patch
          against trunk revision 63c659d.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.server.balancer.TestBalancer
          org.apache.hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot

          The following test timeouts occurred in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.server.namenode.TestBackupNode
          org.apache.hadoop.hdfs.server.namenode.TestSecondaryWebUi

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10234//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10234//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12724284/HDFS-7933.03.patch against trunk revision 63c659d. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.balancer.TestBalancer org.apache.hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot The following test timeouts occurred in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.TestBackupNode org.apache.hadoop.hdfs.server.namenode.TestSecondaryWebUi Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10234//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10234//console This message is automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          It looks like that test run got hurt by some of the classpath problems we've seen. I submitted another test run.

          Show
          cnauroth Chris Nauroth added a comment - It looks like that test run got hurt by some of the classpath problems we've seen. I submitted another test run.
          Hide
          hadoopqa Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12724284/HDFS-7933.03.patch
          against trunk revision 623fd46.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10237//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10237//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12724284/HDFS-7933.03.patch against trunk revision 623fd46. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10237//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10237//console This message is automatically generated.
          Hide
          cnauroth Chris Nauroth added a comment -

          I committed this to trunk and branch-2. Thank you, Xiaoyu.

          Show
          cnauroth Chris Nauroth added a comment - I committed this to trunk and branch-2. Thank you, Xiaoyu.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #7570 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7570/)
          HDFS-7933. fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7570 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7570/ ) HDFS-7933 . fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #161 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/161/)
          HDFS-7933. fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #161 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/161/ ) HDFS-7933 . fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #895 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/895/)
          HDFS-7933. fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #895 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/895/ ) HDFS-7933 . fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2093 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2093/)
          HDFS-7933. fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2093 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2093/ ) HDFS-7933 . fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #152 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/152/)
          HDFS-7933. fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #152 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/152/ ) HDFS-7933 . fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #162 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/162/)
          HDFS-7933. fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #162 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/162/ ) HDFS-7933 . fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2111 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2111/)
          HDFS-7933. fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2111 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2111/ ) HDFS-7933 . fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
          Hide
          aw Allen Wittenauer added a comment -

          Marking this as an incompatible change since it the output of fsck is now different.

          Show
          aw Allen Wittenauer added a comment - Marking this as an incompatible change since it the output of fsck is now different.
          Hide
          zhz Zhe Zhang added a comment -

          Trying to backport this to 2.7. Sorry have to reopen in order to trigger Jenkins.

          Show
          zhz Zhe Zhang added a comment - Trying to backport this to 2.7. Sorry have to reopen in order to trigger Jenkins.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 11m 45s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          +1 mvninstall 9m 30s branch-2.7 passed
          +1 compile 0m 57s branch-2.7 passed with JDK v1.8.0_101
          +1 compile 1m 0s branch-2.7 passed with JDK v1.7.0_101
          +1 checkstyle 0m 28s branch-2.7 passed
          +1 mvnsite 0m 59s branch-2.7 passed
          +1 mvneclipse 0m 18s branch-2.7 passed
          +1 findbugs 3m 0s branch-2.7 passed
          +1 javadoc 1m 0s branch-2.7 passed with JDK v1.8.0_101
          +1 javadoc 1m 39s branch-2.7 passed with JDK v1.7.0_101
          +1 mvninstall 0m 52s the patch passed
          +1 compile 0m 55s the patch passed with JDK v1.8.0_101
          +1 javac 0m 55s the patch passed
          +1 compile 0m 57s the patch passed with JDK v1.7.0_101
          +1 javac 0m 57s the patch passed
          -0 checkstyle 0m 24s hadoop-hdfs-project/hadoop-hdfs: The patch generated 10 new + 274 unchanged - 7 fixed = 284 total (was 281)
          +1 mvnsite 0m 56s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          -1 whitespace 0m 0s The patch has 1691 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          -1 whitespace 0m 45s The patch 221 line(s) with tabs.
          +1 findbugs 3m 21s the patch passed
          +1 javadoc 1m 1s the patch passed with JDK v1.8.0_101
          +1 javadoc 1m 42s the patch passed with JDK v1.7.0_101
          -1 unit 43m 32s hadoop-hdfs in the patch failed with JDK v1.7.0_101.
          -1 asflicense 0m 21s The patch generated 3 ASF License warnings.
          136m 54s



          Reason Tests
          JDK v1.8.0_101 Failed junit tests hadoop.hdfs.TestBlockStoragePolicy
            hadoop.hdfs.TestDatanodeRegistration
            hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
            hadoop.hdfs.server.datanode.TestIncrementalBrVariations
            hadoop.tools.TestJMXGet
          JDK v1.7.0_101 Failed junit tests hadoop.hdfs.web.TestWebHdfsTokens
            hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:c420dfe
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12823759/HDFS-7933-branch-2.7.00.patch
          JIRA Issue HDFS-7933
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux dbe861109574 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2.7 / 7519956
          Default Java 1.7.0_101
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16426/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16426/artifact/patchprocess/whitespace-eol.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16426/artifact/patchprocess/whitespace-tabs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16426/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_101.txt
          JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16426/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/16426/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16426/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 11m 45s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. +1 mvninstall 9m 30s branch-2.7 passed +1 compile 0m 57s branch-2.7 passed with JDK v1.8.0_101 +1 compile 1m 0s branch-2.7 passed with JDK v1.7.0_101 +1 checkstyle 0m 28s branch-2.7 passed +1 mvnsite 0m 59s branch-2.7 passed +1 mvneclipse 0m 18s branch-2.7 passed +1 findbugs 3m 0s branch-2.7 passed +1 javadoc 1m 0s branch-2.7 passed with JDK v1.8.0_101 +1 javadoc 1m 39s branch-2.7 passed with JDK v1.7.0_101 +1 mvninstall 0m 52s the patch passed +1 compile 0m 55s the patch passed with JDK v1.8.0_101 +1 javac 0m 55s the patch passed +1 compile 0m 57s the patch passed with JDK v1.7.0_101 +1 javac 0m 57s the patch passed -0 checkstyle 0m 24s hadoop-hdfs-project/hadoop-hdfs: The patch generated 10 new + 274 unchanged - 7 fixed = 284 total (was 281) +1 mvnsite 0m 56s the patch passed +1 mvneclipse 0m 12s the patch passed -1 whitespace 0m 0s The patch has 1691 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply -1 whitespace 0m 45s The patch 221 line(s) with tabs. +1 findbugs 3m 21s the patch passed +1 javadoc 1m 1s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 42s the patch passed with JDK v1.7.0_101 -1 unit 43m 32s hadoop-hdfs in the patch failed with JDK v1.7.0_101. -1 asflicense 0m 21s The patch generated 3 ASF License warnings. 136m 54s Reason Tests JDK v1.8.0_101 Failed junit tests hadoop.hdfs.TestBlockStoragePolicy   hadoop.hdfs.TestDatanodeRegistration   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   hadoop.hdfs.server.datanode.TestIncrementalBrVariations   hadoop.tools.TestJMXGet JDK v1.7.0_101 Failed junit tests hadoop.hdfs.web.TestWebHdfsTokens   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots Subsystem Report/Notes Docker Image:yetus/hadoop:c420dfe JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12823759/HDFS-7933-branch-2.7.00.patch JIRA Issue HDFS-7933 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux dbe861109574 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / 7519956 Default Java 1.7.0_101 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16426/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16426/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16426/artifact/patchprocess/whitespace-tabs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16426/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_101.txt JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16426/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/16426/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16426/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 13m 12s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          +1 mvninstall 9m 6s branch-2.7 passed
          +1 compile 1m 6s branch-2.7 passed with JDK v1.8.0_101
          +1 compile 1m 7s branch-2.7 passed with JDK v1.7.0_101
          +1 checkstyle 0m 30s branch-2.7 passed
          +1 mvnsite 1m 5s branch-2.7 passed
          +1 mvneclipse 0m 19s branch-2.7 passed
          +1 findbugs 3m 4s branch-2.7 passed
          +1 javadoc 1m 1s branch-2.7 passed with JDK v1.8.0_101
          +1 javadoc 1m 39s branch-2.7 passed with JDK v1.7.0_101
          +1 mvninstall 0m 52s the patch passed
          +1 compile 0m 57s the patch passed with JDK v1.8.0_101
          +1 javac 0m 57s the patch passed
          +1 compile 0m 59s the patch passed with JDK v1.7.0_101
          +1 javac 0m 59s the patch passed
          -0 checkstyle 0m 24s hadoop-hdfs-project/hadoop-hdfs: The patch generated 10 new + 274 unchanged - 7 fixed = 284 total (was 281)
          +1 mvnsite 0m 56s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          -1 whitespace 0m 0s The patch has 1690 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          -1 whitespace 0m 46s The patch 221 line(s) with tabs.
          +1 findbugs 3m 7s the patch passed
          +1 javadoc 0m 55s the patch passed with JDK v1.8.0_101
          +1 javadoc 1m 39s the patch passed with JDK v1.7.0_101
          -1 unit 43m 2s hadoop-hdfs in the patch failed with JDK v1.7.0_101.
          -1 asflicense 0m 22s The patch generated 3 ASF License warnings.
          137m 48s



          Reason Tests
          JDK v1.8.0_101 Failed junit tests hadoop.tools.TestJMXGet
            hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
            hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
          JDK v1.7.0_101 Failed junit tests hadoop.tools.TestJMXGet
            hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:c420dfe
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12823759/HDFS-7933-branch-2.7.00.patch
          JIRA Issue HDFS-7933
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux f9c9d4ebd3b4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2.7 / 7519956
          Default Java 1.7.0_101
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16427/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16427/artifact/patchprocess/whitespace-eol.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16427/artifact/patchprocess/whitespace-tabs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16427/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_101.txt
          JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16427/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/16427/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16427/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 13m 12s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. +1 mvninstall 9m 6s branch-2.7 passed +1 compile 1m 6s branch-2.7 passed with JDK v1.8.0_101 +1 compile 1m 7s branch-2.7 passed with JDK v1.7.0_101 +1 checkstyle 0m 30s branch-2.7 passed +1 mvnsite 1m 5s branch-2.7 passed +1 mvneclipse 0m 19s branch-2.7 passed +1 findbugs 3m 4s branch-2.7 passed +1 javadoc 1m 1s branch-2.7 passed with JDK v1.8.0_101 +1 javadoc 1m 39s branch-2.7 passed with JDK v1.7.0_101 +1 mvninstall 0m 52s the patch passed +1 compile 0m 57s the patch passed with JDK v1.8.0_101 +1 javac 0m 57s the patch passed +1 compile 0m 59s the patch passed with JDK v1.7.0_101 +1 javac 0m 59s the patch passed -0 checkstyle 0m 24s hadoop-hdfs-project/hadoop-hdfs: The patch generated 10 new + 274 unchanged - 7 fixed = 284 total (was 281) +1 mvnsite 0m 56s the patch passed +1 mvneclipse 0m 12s the patch passed -1 whitespace 0m 0s The patch has 1690 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply -1 whitespace 0m 46s The patch 221 line(s) with tabs. +1 findbugs 3m 7s the patch passed +1 javadoc 0m 55s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 39s the patch passed with JDK v1.7.0_101 -1 unit 43m 2s hadoop-hdfs in the patch failed with JDK v1.7.0_101. -1 asflicense 0m 22s The patch generated 3 ASF License warnings. 137m 48s Reason Tests JDK v1.8.0_101 Failed junit tests hadoop.tools.TestJMXGet   hadoop.hdfs.server.namenode.ha.TestPipelinesFailover   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots JDK v1.7.0_101 Failed junit tests hadoop.tools.TestJMXGet   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots Subsystem Report/Notes Docker Image:yetus/hadoop:c420dfe JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12823759/HDFS-7933-branch-2.7.00.patch JIRA Issue HDFS-7933 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux f9c9d4ebd3b4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / 7519956 Default Java 1.7.0_101 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16427/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16427/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16427/artifact/patchprocess/whitespace-tabs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16427/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_101.txt JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16427/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/16427/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16427/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 17m 53s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          +1 mvninstall 6m 40s branch-2.7 passed
          +1 compile 1m 6s branch-2.7 passed with JDK v1.8.0_101
          +1 compile 1m 6s branch-2.7 passed with JDK v1.7.0_101
          +1 checkstyle 0m 27s branch-2.7 passed
          +1 mvnsite 1m 0s branch-2.7 passed
          +1 mvneclipse 0m 14s branch-2.7 passed
          +1 findbugs 3m 1s branch-2.7 passed
          +1 javadoc 1m 12s branch-2.7 passed with JDK v1.8.0_101
          +1 javadoc 1m 43s branch-2.7 passed with JDK v1.7.0_101
          +1 mvninstall 0m 52s the patch passed
          +1 compile 0m 58s the patch passed with JDK v1.8.0_101
          +1 javac 0m 58s the patch passed
          +1 compile 1m 4s the patch passed with JDK v1.7.0_101
          +1 javac 1m 4s the patch passed
          -0 checkstyle 0m 25s hadoop-hdfs-project/hadoop-hdfs: The patch generated 10 new + 274 unchanged - 7 fixed = 284 total (was 281)
          +1 mvnsite 0m 59s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          -1 whitespace 0m 0s The patch has 2320 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
          -1 whitespace 0m 59s The patch 221 line(s) with tabs.
          +1 findbugs 3m 13s the patch passed
          +1 javadoc 1m 0s the patch passed with JDK v1.8.0_101
          +1 javadoc 1m 42s the patch passed with JDK v1.7.0_101
          -1 unit 57m 42s hadoop-hdfs in the patch failed with JDK v1.7.0_101.
          -1 asflicense 0m 19s The patch generated 3 ASF License warnings.
          162m 54s



          Reason Tests
          JDK v1.8.0_101 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
            hadoop.hdfs.server.namenode.TestCacheDirectives
            hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
          JDK v1.7.0_101 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
            hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength
            hadoop.hdfs.server.namenode.TestFileTruncate
            hadoop.hdfs.TestEncryptionZones
            hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:c420dfe
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12823759/HDFS-7933-branch-2.7.00.patch
          JIRA Issue HDFS-7933
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 29da35fca845 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2.7 / 9f5a22f
          Default Java 1.7.0_101
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16442/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16442/artifact/patchprocess/whitespace-eol.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16442/artifact/patchprocess/whitespace-tabs.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16442/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_101.txt
          JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16442/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/16442/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16442/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 17m 53s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. +1 mvninstall 6m 40s branch-2.7 passed +1 compile 1m 6s branch-2.7 passed with JDK v1.8.0_101 +1 compile 1m 6s branch-2.7 passed with JDK v1.7.0_101 +1 checkstyle 0m 27s branch-2.7 passed +1 mvnsite 1m 0s branch-2.7 passed +1 mvneclipse 0m 14s branch-2.7 passed +1 findbugs 3m 1s branch-2.7 passed +1 javadoc 1m 12s branch-2.7 passed with JDK v1.8.0_101 +1 javadoc 1m 43s branch-2.7 passed with JDK v1.7.0_101 +1 mvninstall 0m 52s the patch passed +1 compile 0m 58s the patch passed with JDK v1.8.0_101 +1 javac 0m 58s the patch passed +1 compile 1m 4s the patch passed with JDK v1.7.0_101 +1 javac 1m 4s the patch passed -0 checkstyle 0m 25s hadoop-hdfs-project/hadoop-hdfs: The patch generated 10 new + 274 unchanged - 7 fixed = 284 total (was 281) +1 mvnsite 0m 59s the patch passed +1 mvneclipse 0m 13s the patch passed -1 whitespace 0m 0s The patch has 2320 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply -1 whitespace 0m 59s The patch 221 line(s) with tabs. +1 findbugs 3m 13s the patch passed +1 javadoc 1m 0s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 42s the patch passed with JDK v1.7.0_101 -1 unit 57m 42s hadoop-hdfs in the patch failed with JDK v1.7.0_101. -1 asflicense 0m 19s The patch generated 3 ASF License warnings. 162m 54s Reason Tests JDK v1.8.0_101 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   hadoop.hdfs.server.namenode.TestCacheDirectives   hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot JDK v1.7.0_101 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot Subsystem Report/Notes Docker Image:yetus/hadoop:c420dfe JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12823759/HDFS-7933-branch-2.7.00.patch JIRA Issue HDFS-7933 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 29da35fca845 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / 9f5a22f Default Java 1.7.0_101 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16442/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16442/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16442/artifact/patchprocess/whitespace-tabs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16442/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_101.txt JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16442/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/16442/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16442/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          zhz Zhe Zhang added a comment -

          TestOpenFilesWithSnapshot is broken after HDFS-9696. I verified other reported failures. I just backported the change to branch-2.7.

          Show
          zhz Zhe Zhang added a comment - TestOpenFilesWithSnapshot is broken after HDFS-9696 . I verified other reported failures. I just backported the change to branch-2.7.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          As per versioning rule here,I think, we should not add incompatible changes to dot and minor releases..?

          Show
          brahmareddy Brahma Reddy Battula added a comment - As per versioning rule here ,I think, we should not add incompatible changes to dot and minor releases..?
          Hide
          zhz Zhe Zhang added a comment -

          Brahma Reddy Battula Sorry missed your question. You are right, it was my mistake to backport the incompatible change. Will revert from branch-2.7.

          Show
          zhz Zhe Zhang added a comment - Brahma Reddy Battula Sorry missed your question. You are right, it was my mistake to backport the incompatible change. Will revert from branch-2.7.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          I think, you should revert before 2.7.4 release.

          Show
          brahmareddy Brahma Reddy Battula added a comment - I think, you should revert before 2.7.4 release.
          Hide
          zhz Zhe Zhang added a comment -

          Thanks Brahma Reddy Battula. I filed HDFS-11743.

          Actually can we make this new fsck output configurable, instead of reverting this patch? Differentiating decommissioned and decommissioning is also useful in HDFS-11499.

          Show
          zhz Zhe Zhang added a comment - Thanks Brahma Reddy Battula . I filed HDFS-11743 . Actually can we make this new fsck output configurable, instead of reverting this patch? Differentiating decommissioned and decommissioning is also useful in HDFS-11499 .

            People

            • Assignee:
              xyao Xiaoyu Yao
              Reporter:
              jnp Jitendra Nath Pandey
            • Votes:
              1 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development