Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7933

fsck should also report decommissioning replicas.

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 2.7.4, 3.0.0-alpha1
    • Component/s: namenode
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      The output of hdfs fsck now also contains information about decommissioning replicas.

      Description

      Fsck doesn't count replicas that are on decommissioning nodes. If a block has all replicas on the decommissioning nodes, it will be marked as missing, which is alarming for the admins, although the system will replicate them before nodes are decommissioned.
      Fsck output should also show decommissioning replicas along with the live replicas.

      1. HDFS-7933.00.patch
        8 kB
        Xiaoyu Yao
      2. HDFS-7933.01.patch
        9 kB
        Xiaoyu Yao
      3. HDFS-7933.02.patch
        23 kB
        Xiaoyu Yao
      4. HDFS-7933.03.patch
        23 kB
        Xiaoyu Yao
      5. HDFS-7933-branch-2.7.00.patch
        23 kB
        Zhe Zhang

        Activity

        Hide
        xyao Xiaoyu Yao added a comment -

        Attach a patch that removes the corruption warning for decommissioned replicas and shows them separately with live replicas.

        Show
        xyao Xiaoyu Yao added a comment - Attach a patch that removes the corruption warning for decommissioned replicas and shows them separately with live replicas.
        Hide
        xyao Xiaoyu Yao added a comment -

        Sample output from unit test during decommission:

        Status: HEALTHY
         Total size:	512 B
         Total dirs:	0
         Total files:	1
         Total symlinks:		0
         Total blocks (validated):	1 (avg. block size 512 B)
         Minimally replicated blocks:	1 (100.0 %)
         Over-replicated blocks:	0 (0.0 %)
         Under-replicated blocks:	0 (0.0 %)
         Mis-replicated blocks:		0 (0.0 %)
         Default replication factor:	1
         Average block replication:	2.0
         Corrupt blocks:		0
         Missing replicas:		0
         DecommissioningReplicas:		1 (0.0 %)
         Number of data-nodes:		2
         Number of racks:		2
        FSCK ended at Mon Mar 16 15:04:21 PDT 2015 in 1 milliseconds
        
        
        The filesystem under path '/testfile' is HEALTHY
        
        Show
        xyao Xiaoyu Yao added a comment - Sample output from unit test during decommission: Status: HEALTHY Total size: 512 B Total dirs: 0 Total files: 1 Total symlinks: 0 Total blocks (validated): 1 (avg. block size 512 B) Minimally replicated blocks: 1 (100.0 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks: 0 (0.0 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor: 1 Average block replication: 2.0 Corrupt blocks: 0 Missing replicas: 0 DecommissioningReplicas: 1 (0.0 %) Number of data-nodes: 2 Number of racks: 2 FSCK ended at Mon Mar 16 15:04:21 PDT 2015 in 1 milliseconds The filesystem under path '/testfile' is HEALTHY
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12704903/HDFS-7933.00.patch
        against trunk revision 2681ed9.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.TestClientReportBadBlock

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9905//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9905//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12704903/HDFS-7933.00.patch against trunk revision 2681ed9. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.TestClientReportBadBlock Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/9905//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/9905//console This message is automatically generated.
        Hide
        xyao Xiaoyu Yao added a comment -

        Fix the unit test failure from TestClientReportBadBlock due to fsck output string change.

        Show
        xyao Xiaoyu Yao added a comment - Fix the unit test failure from TestClientReportBadBlock due to fsck output string change.
        Hide
        jnp Jitendra Nath Pandey added a comment -

        Xiaoyu Yao, one of the key issues is that NumReplicas doesn't distinguish between decommissioned or decommissioning replicas, and is initialized with the sum of decommissioning and decommissioned replicas. I think fsck should report decommissioning and decommissioned replicas separately to provide the complete picture.

        Show
        jnp Jitendra Nath Pandey added a comment - Xiaoyu Yao , one of the key issues is that NumReplicas doesn't distinguish between decommissioned or decommissioning replicas, and is initialized with the sum of decommissioning and decommissioned replicas. I think fsck should report decommissioning and decommissioned replicas separately to provide the complete picture.
        Hide
        xyao Xiaoyu Yao added a comment -

        Thanks Jitendra Nath Pandey for reviewing the patch. I've updated the patch based on your feedback.

        Summary of changes:

        1) adding NumberReplicas#decommissioned and NumberReplicas#decommissioning to track the decommissioned and decommissioning replicas, respectively.

        2) deprecating NumberReplicas#decommissionedReplicas() by NumberReplicas#decommissionedAndDecommissioning() to avoid the misleading name.

        3) Display decommissioning and decommissioned replica separately in NamenodeFsck#check().

        Show
        xyao Xiaoyu Yao added a comment - Thanks Jitendra Nath Pandey for reviewing the patch. I've updated the patch based on your feedback. Summary of changes: 1) adding NumberReplicas#decommissioned and NumberReplicas#decommissioning to track the decommissioned and decommissioning replicas, respectively. 2) deprecating NumberReplicas#decommissionedReplicas() by NumberReplicas#decommissionedAndDecommissioning() to avoid the misleading name. 3) Display decommissioning and decommissioned replica separately in NamenodeFsck#check().
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12708365/HDFS-7933.02.patch
        against trunk revision 85dc3c1.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
        org.apache.hadoop.hdfs.server.namenode.TestDefaultBlockPlacementPolicy

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10126//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10126//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12708365/HDFS-7933.02.patch against trunk revision 85dc3c1. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA org.apache.hadoop.hdfs.server.namenode.TestDefaultBlockPlacementPolicy Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10126//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10126//console This message is automatically generated.
        Hide
        cnauroth Chris Nauroth added a comment -

        Hi Xiaoyu. This patch looks good. Here are a few comments:

        1. In BlockManager#chooseSourceDatanode, should the decommissioning counter be incremented by countableReplica, like for the other counters?
                if ((nodesCorrupt != null) && (nodesCorrupt.contains(node)))
                  corrupt += countableReplica;
                else if (node.isDecommissionInProgress()) {
                  decommissioning++;
                } else if (node.isDecommissioned()) {
                  decommissioned += countableReplica;
          
        2. Could the deprecated NumberReplicas#decommissionedReplicas be implemented to forward the call to decommissionedAndDecommissioning? If so, then we can eliminate the decommissionedReplicas member variable.
        3. The new test in TestFsck appears to create a StringBuilder, append data to it, and then never use it for anything. Can it be removed?
        Show
        cnauroth Chris Nauroth added a comment - Hi Xiaoyu. This patch looks good. Here are a few comments: In BlockManager#chooseSourceDatanode , should the decommissioning counter be incremented by countableReplica , like for the other counters? if ((nodesCorrupt != null ) && (nodesCorrupt.contains(node))) corrupt += countableReplica; else if (node.isDecommissionInProgress()) { decommissioning++; } else if (node.isDecommissioned()) { decommissioned += countableReplica; Could the deprecated NumberReplicas#decommissionedReplicas be implemented to forward the call to decommissionedAndDecommissioning ? If so, then we can eliminate the decommissionedReplicas member variable. The new test in TestFsck appears to create a StringBuilder , append data to it, and then never use it for anything. Can it be removed?
        Hide
        xyao Xiaoyu Yao added a comment -

        Thanks Chris for the review. I've updated the patch based on your comments.

        In BlockManager#chooseSourceDatanode, should the decommissioning counter be incremented by countableReplica, like for the other counters?

        Good catch. Fixed.

        Could the deprecated NumberReplicas#decommissionedReplicas be implemented to forward the call to decommissionedAndDecommissioning? If so, then we can eliminate the decommissionedReplicas member variable.

        Agree and fixed.

        The new test in TestFsck appears to create a StringBuilder, append data to it, and then never use it for anything. Can it be removed?

        Fixed.

        Show
        xyao Xiaoyu Yao added a comment - Thanks Chris for the review. I've updated the patch based on your comments. In BlockManager#chooseSourceDatanode, should the decommissioning counter be incremented by countableReplica, like for the other counters? Good catch. Fixed. Could the deprecated NumberReplicas#decommissionedReplicas be implemented to forward the call to decommissionedAndDecommissioning? If so, then we can eliminate the decommissionedReplicas member variable. Agree and fixed. The new test in TestFsck appears to create a StringBuilder, append data to it, and then never use it for anything. Can it be removed? Fixed.
        Hide
        cnauroth Chris Nauroth added a comment -

        +1 for patch v03 pending Jenkins. Thanks for incorporating the feedback. I'm targeting this to 2.8.0 since the release process has begun for 2.7.0.

        Show
        cnauroth Chris Nauroth added a comment - +1 for patch v03 pending Jenkins. Thanks for incorporating the feedback. I'm targeting this to 2.8.0 since the release process has begun for 2.7.0.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12724284/HDFS-7933.03.patch
        against trunk revision 63c659d.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.balancer.TestBalancer
        org.apache.hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot

        The following test timeouts occurred in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.namenode.TestBackupNode
        org.apache.hadoop.hdfs.server.namenode.TestSecondaryWebUi

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10234//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10234//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12724284/HDFS-7933.03.patch against trunk revision 63c659d. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.balancer.TestBalancer org.apache.hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot The following test timeouts occurred in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.TestBackupNode org.apache.hadoop.hdfs.server.namenode.TestSecondaryWebUi Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10234//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10234//console This message is automatically generated.
        Hide
        cnauroth Chris Nauroth added a comment -

        It looks like that test run got hurt by some of the classpath problems we've seen. I submitted another test run.

        Show
        cnauroth Chris Nauroth added a comment - It looks like that test run got hurt by some of the classpath problems we've seen. I submitted another test run.
        Hide
        hadoopqa Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12724284/HDFS-7933.03.patch
        against trunk revision 623fd46.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10237//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10237//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12724284/HDFS-7933.03.patch against trunk revision 623fd46. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10237//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10237//console This message is automatically generated.
        Hide
        cnauroth Chris Nauroth added a comment -

        I committed this to trunk and branch-2. Thank you, Xiaoyu.

        Show
        cnauroth Chris Nauroth added a comment - I committed this to trunk and branch-2. Thank you, Xiaoyu.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #7570 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7570/)
        HDFS-7933. fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7570 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7570/ ) HDFS-7933 . fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #161 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/161/)
        HDFS-7933. fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #161 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/161/ ) HDFS-7933 . fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #895 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/895/)
        HDFS-7933. fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #895 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/895/ ) HDFS-7933 . fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2093 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2093/)
        HDFS-7933. fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2093 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2093/ ) HDFS-7933 . fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #152 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/152/)
        HDFS-7933. fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #152 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/152/ ) HDFS-7933 . fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #162 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/162/)
        HDFS-7933. fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #162 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/162/ ) HDFS-7933 . fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2111 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2111/)
        HDFS-7933. fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2111 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2111/ ) HDFS-7933 . fsck should also report decommissioning replicas. Contributed by Xiaoyu Yao. (cnauroth: rev f8f5887209a7d8e53c0a77abef275cbcaf1f7a5b) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestReadOnlySharedStorage.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
        Hide
        aw Allen Wittenauer added a comment -

        Marking this as an incompatible change since it the output of fsck is now different.

        Show
        aw Allen Wittenauer added a comment - Marking this as an incompatible change since it the output of fsck is now different.
        Hide
        zhz Zhe Zhang added a comment -

        Trying to backport this to 2.7. Sorry have to reopen in order to trigger Jenkins.

        Show
        zhz Zhe Zhang added a comment - Trying to backport this to 2.7. Sorry have to reopen in order to trigger Jenkins.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 11m 45s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
        +1 mvninstall 9m 30s branch-2.7 passed
        +1 compile 0m 57s branch-2.7 passed with JDK v1.8.0_101
        +1 compile 1m 0s branch-2.7 passed with JDK v1.7.0_101
        +1 checkstyle 0m 28s branch-2.7 passed
        +1 mvnsite 0m 59s branch-2.7 passed
        +1 mvneclipse 0m 18s branch-2.7 passed
        +1 findbugs 3m 0s branch-2.7 passed
        +1 javadoc 1m 0s branch-2.7 passed with JDK v1.8.0_101
        +1 javadoc 1m 39s branch-2.7 passed with JDK v1.7.0_101
        +1 mvninstall 0m 52s the patch passed
        +1 compile 0m 55s the patch passed with JDK v1.8.0_101
        +1 javac 0m 55s the patch passed
        +1 compile 0m 57s the patch passed with JDK v1.7.0_101
        +1 javac 0m 57s the patch passed
        -0 checkstyle 0m 24s hadoop-hdfs-project/hadoop-hdfs: The patch generated 10 new + 274 unchanged - 7 fixed = 284 total (was 281)
        +1 mvnsite 0m 56s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        -1 whitespace 0m 0s The patch has 1691 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
        -1 whitespace 0m 45s The patch 221 line(s) with tabs.
        +1 findbugs 3m 21s the patch passed
        +1 javadoc 1m 1s the patch passed with JDK v1.8.0_101
        +1 javadoc 1m 42s the patch passed with JDK v1.7.0_101
        -1 unit 43m 32s hadoop-hdfs in the patch failed with JDK v1.7.0_101.
        -1 asflicense 0m 21s The patch generated 3 ASF License warnings.
        136m 54s



        Reason Tests
        JDK v1.8.0_101 Failed junit tests hadoop.hdfs.TestBlockStoragePolicy
          hadoop.hdfs.TestDatanodeRegistration
          hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
          hadoop.hdfs.server.datanode.TestIncrementalBrVariations
          hadoop.tools.TestJMXGet
        JDK v1.7.0_101 Failed junit tests hadoop.hdfs.web.TestWebHdfsTokens
          hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:c420dfe
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12823759/HDFS-7933-branch-2.7.00.patch
        JIRA Issue HDFS-7933
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux dbe861109574 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision branch-2.7 / 7519956
        Default Java 1.7.0_101
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16426/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16426/artifact/patchprocess/whitespace-eol.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16426/artifact/patchprocess/whitespace-tabs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16426/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_101.txt
        JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16426/testReport/
        asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/16426/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16426/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 11m 45s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. +1 mvninstall 9m 30s branch-2.7 passed +1 compile 0m 57s branch-2.7 passed with JDK v1.8.0_101 +1 compile 1m 0s branch-2.7 passed with JDK v1.7.0_101 +1 checkstyle 0m 28s branch-2.7 passed +1 mvnsite 0m 59s branch-2.7 passed +1 mvneclipse 0m 18s branch-2.7 passed +1 findbugs 3m 0s branch-2.7 passed +1 javadoc 1m 0s branch-2.7 passed with JDK v1.8.0_101 +1 javadoc 1m 39s branch-2.7 passed with JDK v1.7.0_101 +1 mvninstall 0m 52s the patch passed +1 compile 0m 55s the patch passed with JDK v1.8.0_101 +1 javac 0m 55s the patch passed +1 compile 0m 57s the patch passed with JDK v1.7.0_101 +1 javac 0m 57s the patch passed -0 checkstyle 0m 24s hadoop-hdfs-project/hadoop-hdfs: The patch generated 10 new + 274 unchanged - 7 fixed = 284 total (was 281) +1 mvnsite 0m 56s the patch passed +1 mvneclipse 0m 12s the patch passed -1 whitespace 0m 0s The patch has 1691 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply -1 whitespace 0m 45s The patch 221 line(s) with tabs. +1 findbugs 3m 21s the patch passed +1 javadoc 1m 1s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 42s the patch passed with JDK v1.7.0_101 -1 unit 43m 32s hadoop-hdfs in the patch failed with JDK v1.7.0_101. -1 asflicense 0m 21s The patch generated 3 ASF License warnings. 136m 54s Reason Tests JDK v1.8.0_101 Failed junit tests hadoop.hdfs.TestBlockStoragePolicy   hadoop.hdfs.TestDatanodeRegistration   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   hadoop.hdfs.server.datanode.TestIncrementalBrVariations   hadoop.tools.TestJMXGet JDK v1.7.0_101 Failed junit tests hadoop.hdfs.web.TestWebHdfsTokens   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots Subsystem Report/Notes Docker Image:yetus/hadoop:c420dfe JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12823759/HDFS-7933-branch-2.7.00.patch JIRA Issue HDFS-7933 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux dbe861109574 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / 7519956 Default Java 1.7.0_101 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16426/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16426/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16426/artifact/patchprocess/whitespace-tabs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16426/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_101.txt JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16426/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/16426/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16426/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 13m 12s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
        +1 mvninstall 9m 6s branch-2.7 passed
        +1 compile 1m 6s branch-2.7 passed with JDK v1.8.0_101
        +1 compile 1m 7s branch-2.7 passed with JDK v1.7.0_101
        +1 checkstyle 0m 30s branch-2.7 passed
        +1 mvnsite 1m 5s branch-2.7 passed
        +1 mvneclipse 0m 19s branch-2.7 passed
        +1 findbugs 3m 4s branch-2.7 passed
        +1 javadoc 1m 1s branch-2.7 passed with JDK v1.8.0_101
        +1 javadoc 1m 39s branch-2.7 passed with JDK v1.7.0_101
        +1 mvninstall 0m 52s the patch passed
        +1 compile 0m 57s the patch passed with JDK v1.8.0_101
        +1 javac 0m 57s the patch passed
        +1 compile 0m 59s the patch passed with JDK v1.7.0_101
        +1 javac 0m 59s the patch passed
        -0 checkstyle 0m 24s hadoop-hdfs-project/hadoop-hdfs: The patch generated 10 new + 274 unchanged - 7 fixed = 284 total (was 281)
        +1 mvnsite 0m 56s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        -1 whitespace 0m 0s The patch has 1690 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
        -1 whitespace 0m 46s The patch 221 line(s) with tabs.
        +1 findbugs 3m 7s the patch passed
        +1 javadoc 0m 55s the patch passed with JDK v1.8.0_101
        +1 javadoc 1m 39s the patch passed with JDK v1.7.0_101
        -1 unit 43m 2s hadoop-hdfs in the patch failed with JDK v1.7.0_101.
        -1 asflicense 0m 22s The patch generated 3 ASF License warnings.
        137m 48s



        Reason Tests
        JDK v1.8.0_101 Failed junit tests hadoop.tools.TestJMXGet
          hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
          hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
        JDK v1.7.0_101 Failed junit tests hadoop.tools.TestJMXGet
          hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:c420dfe
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12823759/HDFS-7933-branch-2.7.00.patch
        JIRA Issue HDFS-7933
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux f9c9d4ebd3b4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision branch-2.7 / 7519956
        Default Java 1.7.0_101
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16427/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16427/artifact/patchprocess/whitespace-eol.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16427/artifact/patchprocess/whitespace-tabs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16427/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_101.txt
        JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16427/testReport/
        asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/16427/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16427/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 13m 12s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. +1 mvninstall 9m 6s branch-2.7 passed +1 compile 1m 6s branch-2.7 passed with JDK v1.8.0_101 +1 compile 1m 7s branch-2.7 passed with JDK v1.7.0_101 +1 checkstyle 0m 30s branch-2.7 passed +1 mvnsite 1m 5s branch-2.7 passed +1 mvneclipse 0m 19s branch-2.7 passed +1 findbugs 3m 4s branch-2.7 passed +1 javadoc 1m 1s branch-2.7 passed with JDK v1.8.0_101 +1 javadoc 1m 39s branch-2.7 passed with JDK v1.7.0_101 +1 mvninstall 0m 52s the patch passed +1 compile 0m 57s the patch passed with JDK v1.8.0_101 +1 javac 0m 57s the patch passed +1 compile 0m 59s the patch passed with JDK v1.7.0_101 +1 javac 0m 59s the patch passed -0 checkstyle 0m 24s hadoop-hdfs-project/hadoop-hdfs: The patch generated 10 new + 274 unchanged - 7 fixed = 284 total (was 281) +1 mvnsite 0m 56s the patch passed +1 mvneclipse 0m 12s the patch passed -1 whitespace 0m 0s The patch has 1690 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply -1 whitespace 0m 46s The patch 221 line(s) with tabs. +1 findbugs 3m 7s the patch passed +1 javadoc 0m 55s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 39s the patch passed with JDK v1.7.0_101 -1 unit 43m 2s hadoop-hdfs in the patch failed with JDK v1.7.0_101. -1 asflicense 0m 22s The patch generated 3 ASF License warnings. 137m 48s Reason Tests JDK v1.8.0_101 Failed junit tests hadoop.tools.TestJMXGet   hadoop.hdfs.server.namenode.ha.TestPipelinesFailover   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots JDK v1.7.0_101 Failed junit tests hadoop.tools.TestJMXGet   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots Subsystem Report/Notes Docker Image:yetus/hadoop:c420dfe JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12823759/HDFS-7933-branch-2.7.00.patch JIRA Issue HDFS-7933 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux f9c9d4ebd3b4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / 7519956 Default Java 1.7.0_101 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16427/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16427/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16427/artifact/patchprocess/whitespace-tabs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16427/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_101.txt JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16427/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/16427/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16427/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 17m 53s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
        +1 mvninstall 6m 40s branch-2.7 passed
        +1 compile 1m 6s branch-2.7 passed with JDK v1.8.0_101
        +1 compile 1m 6s branch-2.7 passed with JDK v1.7.0_101
        +1 checkstyle 0m 27s branch-2.7 passed
        +1 mvnsite 1m 0s branch-2.7 passed
        +1 mvneclipse 0m 14s branch-2.7 passed
        +1 findbugs 3m 1s branch-2.7 passed
        +1 javadoc 1m 12s branch-2.7 passed with JDK v1.8.0_101
        +1 javadoc 1m 43s branch-2.7 passed with JDK v1.7.0_101
        +1 mvninstall 0m 52s the patch passed
        +1 compile 0m 58s the patch passed with JDK v1.8.0_101
        +1 javac 0m 58s the patch passed
        +1 compile 1m 4s the patch passed with JDK v1.7.0_101
        +1 javac 1m 4s the patch passed
        -0 checkstyle 0m 25s hadoop-hdfs-project/hadoop-hdfs: The patch generated 10 new + 274 unchanged - 7 fixed = 284 total (was 281)
        +1 mvnsite 0m 59s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        -1 whitespace 0m 0s The patch has 2320 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
        -1 whitespace 0m 59s The patch 221 line(s) with tabs.
        +1 findbugs 3m 13s the patch passed
        +1 javadoc 1m 0s the patch passed with JDK v1.8.0_101
        +1 javadoc 1m 42s the patch passed with JDK v1.7.0_101
        -1 unit 57m 42s hadoop-hdfs in the patch failed with JDK v1.7.0_101.
        -1 asflicense 0m 19s The patch generated 3 ASF License warnings.
        162m 54s



        Reason Tests
        JDK v1.8.0_101 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
          hadoop.hdfs.server.namenode.TestCacheDirectives
          hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
        JDK v1.7.0_101 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
          hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength
          hadoop.hdfs.server.namenode.TestFileTruncate
          hadoop.hdfs.TestEncryptionZones
          hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:c420dfe
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12823759/HDFS-7933-branch-2.7.00.patch
        JIRA Issue HDFS-7933
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 29da35fca845 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision branch-2.7 / 9f5a22f
        Default Java 1.7.0_101
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16442/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16442/artifact/patchprocess/whitespace-eol.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16442/artifact/patchprocess/whitespace-tabs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16442/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_101.txt
        JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16442/testReport/
        asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/16442/artifact/patchprocess/patch-asflicense-problems.txt
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16442/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 17m 53s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. +1 mvninstall 6m 40s branch-2.7 passed +1 compile 1m 6s branch-2.7 passed with JDK v1.8.0_101 +1 compile 1m 6s branch-2.7 passed with JDK v1.7.0_101 +1 checkstyle 0m 27s branch-2.7 passed +1 mvnsite 1m 0s branch-2.7 passed +1 mvneclipse 0m 14s branch-2.7 passed +1 findbugs 3m 1s branch-2.7 passed +1 javadoc 1m 12s branch-2.7 passed with JDK v1.8.0_101 +1 javadoc 1m 43s branch-2.7 passed with JDK v1.7.0_101 +1 mvninstall 0m 52s the patch passed +1 compile 0m 58s the patch passed with JDK v1.8.0_101 +1 javac 0m 58s the patch passed +1 compile 1m 4s the patch passed with JDK v1.7.0_101 +1 javac 1m 4s the patch passed -0 checkstyle 0m 25s hadoop-hdfs-project/hadoop-hdfs: The patch generated 10 new + 274 unchanged - 7 fixed = 284 total (was 281) +1 mvnsite 0m 59s the patch passed +1 mvneclipse 0m 13s the patch passed -1 whitespace 0m 0s The patch has 2320 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply -1 whitespace 0m 59s The patch 221 line(s) with tabs. +1 findbugs 3m 13s the patch passed +1 javadoc 1m 0s the patch passed with JDK v1.8.0_101 +1 javadoc 1m 42s the patch passed with JDK v1.7.0_101 -1 unit 57m 42s hadoop-hdfs in the patch failed with JDK v1.7.0_101. -1 asflicense 0m 19s The patch generated 3 ASF License warnings. 162m 54s Reason Tests JDK v1.8.0_101 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   hadoop.hdfs.server.namenode.TestCacheDirectives   hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot JDK v1.7.0_101 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot Subsystem Report/Notes Docker Image:yetus/hadoop:c420dfe JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12823759/HDFS-7933-branch-2.7.00.patch JIRA Issue HDFS-7933 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 29da35fca845 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / 9f5a22f Default Java 1.7.0_101 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_101 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16442/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16442/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/16442/artifact/patchprocess/whitespace-tabs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16442/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_101.txt JDK v1.7.0_101 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16442/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/16442/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16442/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        zhz Zhe Zhang added a comment -

        TestOpenFilesWithSnapshot is broken after HDFS-9696. I verified other reported failures. I just backported the change to branch-2.7.

        Show
        zhz Zhe Zhang added a comment - TestOpenFilesWithSnapshot is broken after HDFS-9696 . I verified other reported failures. I just backported the change to branch-2.7.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        As per versioning rule here,I think, we should not add incompatible changes to dot and minor releases..?

        Show
        brahmareddy Brahma Reddy Battula added a comment - As per versioning rule here ,I think, we should not add incompatible changes to dot and minor releases..?

          People

          • Assignee:
            xyao Xiaoyu Yao
            Reporter:
            jnp Jitendra Nath Pandey
          • Votes:
            1 Vote for this issue
            Watchers:
            12 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development