Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11445

FSCK shows overall health stauts as corrupt even one replica is corrupt

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 2.7.4, 3.0.0-alpha4, 2.8.2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In the following scenario,FSCK shows overall health status as corrupt even it's has one good replica.

      1. Create file with 2 RF.
      2. Shutdown one DN
      3. Append to file again.
      4. Restart the DN
      5. After block report, check Fsck

      1. HDFS-11445.patch
        4 kB
        Brahma Reddy Battula
      2. HDFS-11445-002.patch
        8 kB
        Brahma Reddy Battula
      3. HDFS-11445-003.patch
        10 kB
        Brahma Reddy Battula
      4. HDFS-11445-004.patch
        10 kB
        Brahma Reddy Battula
      5. HDFS-11445-005.patch
        10 kB
        Brahma Reddy Battula
      6. HDFS-11445-branch-2.7.patch
        11 kB
        Brahma Reddy Battula
      7. HDFS-11445-branch-2.7-002.patch
        11 kB
        Brahma Reddy Battula
      8. HDFS-11445-branch-2.patch
        10 kB
        Brahma Reddy Battula

        Activity

        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Uploaded the patch.Kindly Review.

        Show
        brahmareddy Brahma Reddy Battula added a comment - Uploaded the patch.Kindly Review.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 23s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 13m 38s trunk passed
        +1 compile 0m 46s trunk passed
        +1 checkstyle 0m 35s trunk passed
        +1 mvnsite 0m 52s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 46s trunk passed
        +1 javadoc 0m 40s trunk passed
        +1 mvninstall 0m 47s the patch passed
        +1 compile 0m 45s the patch passed
        +1 javac 0m 45s the patch passed
        +1 checkstyle 0m 33s the patch passed
        +1 mvnsite 0m 48s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 1s The patch has no whitespace issues.
        +1 findbugs 1m 51s the patch passed
        +1 javadoc 0m 39s the patch passed
        -1 unit 74m 31s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 29s The patch does not generate ASF License warnings.
        100m 47s



        Reason Tests
        Failed junit tests hadoop.hdfs.TestMaintenanceState
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration
          hadoop.hdfs.server.datanode.TestDirectoryScanner
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11445
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12854146/HDFS-11445.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 762077f73dda 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 13d4bcf
        Default Java 1.8.0_121
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18423/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18423/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18423/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 23s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 13m 38s trunk passed +1 compile 0m 46s trunk passed +1 checkstyle 0m 35s trunk passed +1 mvnsite 0m 52s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 46s trunk passed +1 javadoc 0m 40s trunk passed +1 mvninstall 0m 47s the patch passed +1 compile 0m 45s the patch passed +1 javac 0m 45s the patch passed +1 checkstyle 0m 33s the patch passed +1 mvnsite 0m 48s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 1s The patch has no whitespace issues. +1 findbugs 1m 51s the patch passed +1 javadoc 0m 39s the patch passed -1 unit 74m 31s hadoop-hdfs in the patch failed. +1 asflicense 0m 29s The patch does not generate ASF License warnings. 100m 47s Reason Tests Failed junit tests hadoop.hdfs.TestMaintenanceState   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration   hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11445 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12854146/HDFS-11445.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 762077f73dda 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 13d4bcf Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18423/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18423/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18423/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Test failures are unrelated..Kindly review.

        Show
        brahmareddy Brahma Reddy Battula added a comment - Test failures are unrelated..Kindly review.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Hi Jing Zhao/Kihwal Lee/Tsz Wo Nicholas Sze

        If you get chance, can you please review..?

        Show
        brahmareddy Brahma Reddy Battula added a comment - Hi Jing Zhao / Kihwal Lee / Tsz Wo Nicholas Sze If you get chance, can you please review..?
        Hide
        jingzhao Jing Zhao added a comment -

        Thanks for working on this, Brahma Reddy Battula!

        I think you just find a scenario that the following inconsistency happens:

        BlockManager#createLocatedBlock
            NumberReplicas numReplicas = countNodes(blk);
            final int numCorruptNodes = numReplicas.corruptReplicas();
            final int numCorruptReplicas = corruptReplicas.numCorruptReplicas(blk);
            if (numCorruptNodes != numCorruptReplicas) {
              LOG.warn("Inconsistent number of corrupt replicas for "
                  + blk + " blockMap has " + numCorruptNodes
                  + " but corrupt replicas map has " + numCorruptReplicas);
            }
        

        I also did some debugging using your unit test. Looks like the root cause for this inconsistency is: BlockInfo#setGenerationStampAndVerifyReplicas may remove a datanode storage from the block's storage list, but still leave the storage in the CorruptReplicasMap.

        This inconsistency later can be fixed automatically, e.g., by a full block report. But maybe we should consider using BlockManager#removeStoredBlock(BlockInfo, DatanodeDescriptor) to remove all the records related to the block-dn pair.

        Show
        jingzhao Jing Zhao added a comment - Thanks for working on this, Brahma Reddy Battula ! I think you just find a scenario that the following inconsistency happens: BlockManager#createLocatedBlock NumberReplicas numReplicas = countNodes(blk); final int numCorruptNodes = numReplicas.corruptReplicas(); final int numCorruptReplicas = corruptReplicas.numCorruptReplicas(blk); if (numCorruptNodes != numCorruptReplicas) { LOG.warn( "Inconsistent number of corrupt replicas for " + blk + " blockMap has " + numCorruptNodes + " but corrupt replicas map has " + numCorruptReplicas); } I also did some debugging using your unit test. Looks like the root cause for this inconsistency is: BlockInfo#setGenerationStampAndVerifyReplicas may remove a datanode storage from the block's storage list, but still leave the storage in the CorruptReplicasMap. This inconsistency later can be fixed automatically, e.g., by a full block report. But maybe we should consider using BlockManager#removeStoredBlock(BlockInfo, DatanodeDescriptor) to remove all the records related to the block-dn pair.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Jing Zhao thanks a lot for your review..

        This inconsistency later can be fixed automatically, e.g., by a full block report. But maybe we should consider using BlockManager#removeStoredBlock(BlockInfo, DatanodeDescriptor) to remove all the records related to the block-dn pair

        Yes,Updated the patch. Kindly review.

        Show
        brahmareddy Brahma Reddy Battula added a comment - Jing Zhao thanks a lot for your review.. This inconsistency later can be fixed automatically, e.g., by a full block report. But maybe we should consider using BlockManager#removeStoredBlock(BlockInfo, DatanodeDescriptor) to remove all the records related to the block-dn pair Yes,Updated the patch. Kindly review.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 15m 10s trunk passed
        +1 compile 0m 48s trunk passed
        +1 checkstyle 0m 40s trunk passed
        +1 mvnsite 0m 57s trunk passed
        +1 mvneclipse 0m 15s trunk passed
        -1 findbugs 2m 2s hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings.
        +1 javadoc 0m 47s trunk passed
        +1 mvninstall 0m 57s the patch passed
        +1 compile 0m 54s the patch passed
        +1 javac 0m 54s the patch passed
        -0 checkstyle 0m 39s hadoop-hdfs-project/hadoop-hdfs: The patch generated 5 new + 310 unchanged - 1 fixed = 315 total (was 311)
        +1 mvnsite 1m 0s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 2m 0s the patch passed
        +1 javadoc 0m 43s the patch passed
        -1 unit 73m 46s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 19s The patch does not generate ASF License warnings.
        102m 56s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue HDFS-11445
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12868260/HDFS-11445-002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 722940c3b59d 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / c48f297
        Default Java 1.8.0_131
        findbugs v3.1.0-RC1
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19447/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19447/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/19447/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19447/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19447/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 15m 10s trunk passed +1 compile 0m 48s trunk passed +1 checkstyle 0m 40s trunk passed +1 mvnsite 0m 57s trunk passed +1 mvneclipse 0m 15s trunk passed -1 findbugs 2m 2s hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings. +1 javadoc 0m 47s trunk passed +1 mvninstall 0m 57s the patch passed +1 compile 0m 54s the patch passed +1 javac 0m 54s the patch passed -0 checkstyle 0m 39s hadoop-hdfs-project/hadoop-hdfs: The patch generated 5 new + 310 unchanged - 1 fixed = 315 total (was 311) +1 mvnsite 1m 0s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 0s the patch passed +1 javadoc 0m 43s the patch passed -1 unit 73m 46s hadoop-hdfs in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 102m 56s Reason Tests Failed junit tests hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-11445 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12868260/HDFS-11445-002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 722940c3b59d 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c48f297 Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/19447/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19447/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/19447/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19447/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19447/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        jingzhao Jing Zhao added a comment -

        Thanks for the patch, Brahma Reddy Battula. For the current patch, instead of passing BlockManager to commitBlock and setGenerationStampAnderifyReplcias, why not letting these methods return the list of stale replicas and then removing these stored blockInfo in BlockManager?

        Show
        jingzhao Jing Zhao added a comment - Thanks for the patch, Brahma Reddy Battula . For the current patch, instead of passing BlockManager to commitBlock and setGenerationStampAnderifyReplcias , why not letting these methods return the list of stale replicas and then removing these stored blockInfo in BlockManager?
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        why not letting these methods return the list of stale replicas and then removing these stored blockInfo in BlockManager?

        Make sense, Updated the patch, kindly review.

        Show
        brahmareddy Brahma Reddy Battula added a comment - why not letting these methods return the list of stale replicas and then removing these stored blockInfo in BlockManager? Make sense, Updated the patch, kindly review.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 25s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 14m 4s trunk passed
        +1 compile 0m 49s trunk passed
        +1 checkstyle 0m 40s trunk passed
        +1 mvnsite 0m 53s trunk passed
        +1 mvneclipse 0m 15s trunk passed
        +1 findbugs 1m 42s trunk passed
        +1 javadoc 0m 41s trunk passed
        +1 mvninstall 0m 50s the patch passed
        +1 compile 0m 45s the patch passed
        +1 javac 0m 45s the patch passed
        -0 checkstyle 0m 36s hadoop-hdfs-project/hadoop-hdfs: The patch generated 3 new + 309 unchanged - 2 fixed = 312 total (was 311)
        +1 mvnsite 0m 52s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 46s the patch passed
        +1 javadoc 0m 39s the patch passed
        -1 unit 68m 59s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 21s The patch does not generate ASF License warnings.
        95m 55s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080
          hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure
        Timed out junit tests org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue HDFS-11445
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12868531/HDFS-11445-003.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 5ab1f9862b5c 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 035d468
        Default Java 1.8.0_131
        findbugs v3.1.0-RC1
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19469/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/19469/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19469/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19469/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 25s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 14m 4s trunk passed +1 compile 0m 49s trunk passed +1 checkstyle 0m 40s trunk passed +1 mvnsite 0m 53s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 42s trunk passed +1 javadoc 0m 41s trunk passed +1 mvninstall 0m 50s the patch passed +1 compile 0m 45s the patch passed +1 javac 0m 45s the patch passed -0 checkstyle 0m 36s hadoop-hdfs-project/hadoop-hdfs: The patch generated 3 new + 309 unchanged - 2 fixed = 312 total (was 311) +1 mvnsite 0m 52s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 46s the patch passed +1 javadoc 0m 39s the patch passed -1 unit 68m 59s hadoop-hdfs in the patch failed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 95m 55s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080   hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure Timed out junit tests org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-11445 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12868531/HDFS-11445-003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 5ab1f9862b5c 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 035d468 Default Java 1.8.0_131 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19469/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/19469/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19469/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19469/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        jingzhao Jing Zhao added a comment -

        Thanks for updating the patch, Brahma Reddy Battula. The patch looks good to me. +1 after fixing the new checkstyle warnings.

        Show
        jingzhao Jing Zhao added a comment - Thanks for updating the patch, Brahma Reddy Battula . The patch looks good to me. +1 after fixing the new checkstyle warnings.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Jing Zhao thanks for review..

        Updated the patch to Address the check-style.

        Show
        brahmareddy Brahma Reddy Battula added a comment - Jing Zhao thanks for review.. Updated the patch to Address the check-style .
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 18s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 15m 43s trunk passed
        +1 compile 0m 47s trunk passed
        +1 checkstyle 0m 40s trunk passed
        +1 mvnsite 0m 52s trunk passed
        +1 mvneclipse 0m 15s trunk passed
        +1 findbugs 1m 40s trunk passed
        +1 javadoc 0m 38s trunk passed
        +1 mvninstall 0m 48s the patch passed
        +1 compile 0m 44s the patch passed
        +1 javac 0m 44s the patch passed
        -0 checkstyle 0m 36s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 307 unchanged - 2 fixed = 308 total (was 309)
        +1 mvnsite 0m 49s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 45s the patch passed
        +1 javadoc 0m 36s the patch passed
        -1 unit 63m 3s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 19s The patch does not generate ASF License warnings.
        91m 1s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
          hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue HDFS-11445
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12869154/HDFS-11445-004.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 69ae63eaa026 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / e135c0b
        Default Java 1.8.0_131
        findbugs v3.1.0-RC1
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19527/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/19527/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19527/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19527/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 15m 43s trunk passed +1 compile 0m 47s trunk passed +1 checkstyle 0m 40s trunk passed +1 mvnsite 0m 52s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 40s trunk passed +1 javadoc 0m 38s trunk passed +1 mvninstall 0m 48s the patch passed +1 compile 0m 44s the patch passed +1 javac 0m 44s the patch passed -0 checkstyle 0m 36s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 307 unchanged - 2 fixed = 308 total (was 309) +1 mvnsite 0m 49s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 45s the patch passed +1 javadoc 0m 36s the patch passed -1 unit 63m 3s hadoop-hdfs in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 91m 1s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestNamenodeCapacityReport   hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-11445 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12869154/HDFS-11445-004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 69ae63eaa026 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e135c0b Default Java 1.8.0_131 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19527/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/19527/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19527/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19527/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Uploaded the branch-2 which are having minimal conflicts,As EC is not present.

        Show
        brahmareddy Brahma Reddy Battula added a comment - Uploaded the branch-2 which are having minimal conflicts,As EC is not present.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 15m 9s trunk passed
        +1 compile 1m 0s trunk passed
        +1 checkstyle 0m 40s trunk passed
        +1 mvnsite 1m 5s trunk passed
        +1 mvneclipse 0m 15s trunk passed
        +1 findbugs 1m 56s trunk passed
        +1 javadoc 0m 46s trunk passed
        +1 mvninstall 1m 0s the patch passed
        +1 compile 0m 57s the patch passed
        +1 javac 0m 57s the patch passed
        +1 checkstyle 0m 40s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 307 unchanged - 2 fixed = 307 total (was 309)
        +1 mvnsite 0m 53s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 46s the patch passed
        +1 javadoc 0m 39s the patch passed
        -1 unit 68m 57s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 20s The patch does not generate ASF License warnings.
        97m 50s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure090
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure010



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue HDFS-11445
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12869160/HDFS-11445-005.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux cc40f94ddbe2 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / c80d3bc
        Default Java 1.8.0_131
        findbugs v3.1.0-RC1
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/19531/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19531/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19531/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 15m 9s trunk passed +1 compile 1m 0s trunk passed +1 checkstyle 0m 40s trunk passed +1 mvnsite 1m 5s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 56s trunk passed +1 javadoc 0m 46s trunk passed +1 mvninstall 1m 0s the patch passed +1 compile 0m 57s the patch passed +1 javac 0m 57s the patch passed +1 checkstyle 0m 40s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 307 unchanged - 2 fixed = 307 total (was 309) +1 mvnsite 0m 53s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 46s the patch passed +1 javadoc 0m 39s the patch passed -1 unit 68m 57s hadoop-hdfs in the patch failed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 97m 50s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure090   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure010 Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-11445 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12869160/HDFS-11445-005.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux cc40f94ddbe2 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c80d3bc Default Java 1.8.0_131 findbugs v3.1.0-RC1 unit https://builds.apache.org/job/PreCommit-HDFS-Build/19531/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19531/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19531/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 13m 26s trunk passed
        +1 compile 0m 52s trunk passed
        +1 checkstyle 0m 41s trunk passed
        +1 mvnsite 0m 57s trunk passed
        +1 mvneclipse 0m 15s trunk passed
        +1 findbugs 1m 45s trunk passed
        +1 javadoc 0m 46s trunk passed
        +1 mvninstall 0m 52s the patch passed
        +1 compile 0m 50s the patch passed
        +1 javac 0m 50s the patch passed
        +1 checkstyle 0m 37s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 307 unchanged - 2 fixed = 307 total (was 309)
        +1 mvnsite 0m 53s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 52s the patch passed
        +1 javadoc 0m 38s the patch passed
        -1 unit 75m 55s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 19s The patch does not generate ASF License warnings.
        102m 29s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
          hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:14b5c93
        JIRA Issue HDFS-11445
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12869160/HDFS-11445-005.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 04cdd2706ce6 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / c80d3bc
        Default Java 1.8.0_131
        findbugs v3.1.0-RC1
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/19530/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19530/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19530/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 13m 26s trunk passed +1 compile 0m 52s trunk passed +1 checkstyle 0m 41s trunk passed +1 mvnsite 0m 57s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 45s trunk passed +1 javadoc 0m 46s trunk passed +1 mvninstall 0m 52s the patch passed +1 compile 0m 50s the patch passed +1 javac 0m 50s the patch passed +1 checkstyle 0m 37s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 307 unchanged - 2 fixed = 307 total (was 309) +1 mvnsite 0m 53s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 52s the patch passed +1 javadoc 0m 38s the patch passed -1 unit 75m 55s hadoop-hdfs in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 102m 29s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-11445 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12869160/HDFS-11445-005.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 04cdd2706ce6 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c80d3bc Default Java 1.8.0_131 findbugs v3.1.0-RC1 unit https://builds.apache.org/job/PreCommit-HDFS-Build/19530/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19530/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19530/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 20s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 6s branch-2 passed
        +1 compile 0m 46s branch-2 passed with JDK v1.8.0_131
        +1 compile 0m 47s branch-2 passed with JDK v1.7.0_131
        +1 checkstyle 0m 32s branch-2 passed
        +1 mvnsite 0m 56s branch-2 passed
        +1 mvneclipse 0m 15s branch-2 passed
        +1 findbugs 2m 8s branch-2 passed
        +1 javadoc 0m 40s branch-2 passed with JDK v1.8.0_131
        +1 javadoc 1m 0s branch-2 passed with JDK v1.7.0_131
        +1 mvninstall 0m 49s the patch passed
        +1 compile 0m 47s the patch passed with JDK v1.8.0_131
        +1 javac 0m 47s the patch passed
        +1 compile 0m 48s the patch passed with JDK v1.7.0_131
        +1 javac 0m 48s the patch passed
        +1 checkstyle 0m 33s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 339 unchanged - 2 fixed = 339 total (was 341)
        +1 mvnsite 0m 58s the patch passed
        +1 mvneclipse 0m 15s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 2m 26s the patch passed
        +1 javadoc 0m 41s the patch passed with JDK v1.8.0_131
        +1 javadoc 1m 2s the patch passed with JDK v1.7.0_131
        -1 unit 52m 24s hadoop-hdfs in the patch failed with JDK v1.7.0_131.
        +1 asflicense 0m 22s The patch does not generate ASF License warnings.
        144m 39s



        Reason Tests
        JDK v1.8.0_131 Timed out junit tests org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
          org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
        JDK v1.7.0_131 Failed junit tests hadoop.metrics2.sink.TestRollingFileSystemSinkWithSecureHdfs
          hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithUpgradeDomain



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:8515d35
        JIRA Issue HDFS-11445
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12869161/HDFS-11445-branch-2.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 4b6cb04a38ee 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision branch-2 / fe185e2
        Default Java 1.7.0_131
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_131 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_131
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/19532/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_131.txt
        JDK v1.7.0_131 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19532/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19532/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 6s branch-2 passed +1 compile 0m 46s branch-2 passed with JDK v1.8.0_131 +1 compile 0m 47s branch-2 passed with JDK v1.7.0_131 +1 checkstyle 0m 32s branch-2 passed +1 mvnsite 0m 56s branch-2 passed +1 mvneclipse 0m 15s branch-2 passed +1 findbugs 2m 8s branch-2 passed +1 javadoc 0m 40s branch-2 passed with JDK v1.8.0_131 +1 javadoc 1m 0s branch-2 passed with JDK v1.7.0_131 +1 mvninstall 0m 49s the patch passed +1 compile 0m 47s the patch passed with JDK v1.8.0_131 +1 javac 0m 47s the patch passed +1 compile 0m 48s the patch passed with JDK v1.7.0_131 +1 javac 0m 48s the patch passed +1 checkstyle 0m 33s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 339 unchanged - 2 fixed = 339 total (was 341) +1 mvnsite 0m 58s the patch passed +1 mvneclipse 0m 15s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 26s the patch passed +1 javadoc 0m 41s the patch passed with JDK v1.8.0_131 +1 javadoc 1m 2s the patch passed with JDK v1.7.0_131 -1 unit 52m 24s hadoop-hdfs in the patch failed with JDK v1.7.0_131. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 144m 39s Reason Tests JDK v1.8.0_131 Timed out junit tests org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure JDK v1.7.0_131 Failed junit tests hadoop.metrics2.sink.TestRollingFileSystemSinkWithSecureHdfs   hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithUpgradeDomain Subsystem Report/Notes Docker Image:yetus/hadoop:8515d35 JIRA Issue HDFS-11445 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12869161/HDFS-11445-branch-2.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 4b6cb04a38ee 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / fe185e2 Default Java 1.7.0_131 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_131 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_131 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/19532/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_131.txt JDK v1.7.0_131 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19532/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19532/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        test failures are unrelated.Jing Zhao can you please take look branch-2 also..I will commit once after your final review.

        Show
        brahmareddy Brahma Reddy Battula added a comment - test failures are unrelated. Jing Zhao can you please take look branch-2 also..I will commit once after your final review.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Uploading the branch-2.7 patch.

        Show
        brahmareddy Brahma Reddy Battula added a comment - Uploading the branch-2.7 patch.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 30s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 8m 11s branch-2.7 passed
        +1 compile 1m 2s branch-2.7 passed with JDK v1.8.0_131
        +1 compile 1m 4s branch-2.7 passed with JDK v1.7.0_131
        +1 checkstyle 0m 30s branch-2.7 passed
        +1 mvnsite 0m 59s branch-2.7 passed
        +1 mvneclipse 0m 16s branch-2.7 passed
        +1 findbugs 2m 57s branch-2.7 passed
        +1 javadoc 0m 58s branch-2.7 passed with JDK v1.8.0_131
        +1 javadoc 1m 45s branch-2.7 passed with JDK v1.7.0_131
        +1 mvninstall 0m 50s the patch passed
        +1 compile 0m 55s the patch passed with JDK v1.8.0_131
        +1 javac 0m 55s the patch passed
        +1 compile 0m 58s the patch passed with JDK v1.7.0_131
        +1 javac 0m 58s the patch passed
        -0 checkstyle 0m 27s hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 561 unchanged - 1 fixed = 563 total (was 562)
        +1 mvnsite 0m 55s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        -1 whitespace 0m 0s The patch has 3198 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
        -1 whitespace 1m 22s The patch 70 line(s) with tabs.
        +1 findbugs 3m 5s the patch passed
        +1 javadoc 0m 56s the patch passed with JDK v1.8.0_131
        +1 javadoc 1m 39s the patch passed with JDK v1.7.0_131
        -1 unit 45m 48s hadoop-hdfs in the patch failed with JDK v1.7.0_131.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        126m 31s



        Reason Tests
        JDK v1.8.0_131 Failed junit tests hadoop.hdfs.server.namenode.TestFileTruncate
          hadoop.hdfs.web.TestWebHdfsTokens
          hadoop.hdfs.server.datanode.TestBlockScanner
          hadoop.hdfs.server.namenode.TestCacheDirectives
          hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
        JDK v1.7.0_131 Failed junit tests hadoop.hdfs.server.namenode.TestDecommissioningStatus
          hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:c420dfe
        JIRA Issue HDFS-11445
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12869205/HDFS-11445-branch-2.7.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 6578bd1243a0 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision branch-2.7 / 8a0e5e3
        Default Java 1.7.0_131
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_131 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_131
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19537/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/19537/artifact/patchprocess/whitespace-eol.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/19537/artifact/patchprocess/whitespace-tabs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/19537/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_131.txt
        JDK v1.7.0_131 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19537/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19537/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 30s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 11s branch-2.7 passed +1 compile 1m 2s branch-2.7 passed with JDK v1.8.0_131 +1 compile 1m 4s branch-2.7 passed with JDK v1.7.0_131 +1 checkstyle 0m 30s branch-2.7 passed +1 mvnsite 0m 59s branch-2.7 passed +1 mvneclipse 0m 16s branch-2.7 passed +1 findbugs 2m 57s branch-2.7 passed +1 javadoc 0m 58s branch-2.7 passed with JDK v1.8.0_131 +1 javadoc 1m 45s branch-2.7 passed with JDK v1.7.0_131 +1 mvninstall 0m 50s the patch passed +1 compile 0m 55s the patch passed with JDK v1.8.0_131 +1 javac 0m 55s the patch passed +1 compile 0m 58s the patch passed with JDK v1.7.0_131 +1 javac 0m 58s the patch passed -0 checkstyle 0m 27s hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 561 unchanged - 1 fixed = 563 total (was 562) +1 mvnsite 0m 55s the patch passed +1 mvneclipse 0m 12s the patch passed -1 whitespace 0m 0s The patch has 3198 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply -1 whitespace 1m 22s The patch 70 line(s) with tabs. +1 findbugs 3m 5s the patch passed +1 javadoc 0m 56s the patch passed with JDK v1.8.0_131 +1 javadoc 1m 39s the patch passed with JDK v1.7.0_131 -1 unit 45m 48s hadoop-hdfs in the patch failed with JDK v1.7.0_131. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 126m 31s Reason Tests JDK v1.8.0_131 Failed junit tests hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.hdfs.web.TestWebHdfsTokens   hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.server.namenode.TestCacheDirectives   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots JDK v1.7.0_131 Failed junit tests hadoop.hdfs.server.namenode.TestDecommissioningStatus   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots Subsystem Report/Notes Docker Image:yetus/hadoop:c420dfe JIRA Issue HDFS-11445 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12869205/HDFS-11445-branch-2.7.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 6578bd1243a0 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / 8a0e5e3 Default Java 1.7.0_131 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_131 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_131 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19537/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/19537/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/19537/artifact/patchprocess/whitespace-tabs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/19537/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_131.txt JDK v1.7.0_131 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19537/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19537/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Updated the patch to fix the check-style ('if' construct must use '{}'s,could have fixed while committing. ),import length can be ignored.

        Show
        brahmareddy Brahma Reddy Battula added a comment - Updated the patch to fix the check-style ('if' construct must use '{}'s,could have fixed while committing. ), import length can be ignored.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 11m 40s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 3s branch-2.7 passed
        +1 compile 0m 59s branch-2.7 passed with JDK v1.8.0_131
        +1 compile 1m 1s branch-2.7 passed with JDK v1.7.0_131
        +1 checkstyle 0m 29s branch-2.7 passed
        +1 mvnsite 0m 57s branch-2.7 passed
        +1 mvneclipse 0m 14s branch-2.7 passed
        +1 findbugs 2m 57s branch-2.7 passed
        +1 javadoc 0m 59s branch-2.7 passed with JDK v1.8.0_131
        +1 javadoc 1m 45s branch-2.7 passed with JDK v1.7.0_131
        +1 mvninstall 0m 53s the patch passed
        +1 compile 0m 59s the patch passed with JDK v1.8.0_131
        +1 javac 0m 59s the patch passed
        +1 compile 1m 0s the patch passed with JDK v1.7.0_131
        +1 javac 1m 0s the patch passed
        -0 checkstyle 0m 27s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 561 unchanged - 1 fixed = 562 total (was 562)
        +1 mvnsite 0m 57s the patch passed
        +1 mvneclipse 0m 12s the patch passed
        -1 whitespace 0m 0s The patch has 3522 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
        -1 whitespace 1m 27s The patch 70 line(s) with tabs.
        +1 findbugs 3m 13s the patch passed
        +1 javadoc 0m 56s the patch passed with JDK v1.8.0_131
        +1 javadoc 1m 43s the patch passed with JDK v1.7.0_131
        -1 unit 44m 11s hadoop-hdfs in the patch failed with JDK v1.7.0_131.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        132m 46s



        Reason Tests
        JDK v1.8.0_131 Failed junit tests hadoop.hdfs.web.TestHttpsFileSystem
          hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
        JDK v1.7.0_131 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:c420dfe
        JIRA Issue HDFS-11445
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12869243/HDFS-11445-branch-2.7-002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 2cc707c48a62 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision branch-2.7 / 8a0e5e3
        Default Java 1.7.0_131
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_131 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_131
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19541/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/19541/artifact/patchprocess/whitespace-eol.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/19541/artifact/patchprocess/whitespace-tabs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/19541/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_131.txt
        JDK v1.7.0_131 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19541/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19541/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 11m 40s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 3s branch-2.7 passed +1 compile 0m 59s branch-2.7 passed with JDK v1.8.0_131 +1 compile 1m 1s branch-2.7 passed with JDK v1.7.0_131 +1 checkstyle 0m 29s branch-2.7 passed +1 mvnsite 0m 57s branch-2.7 passed +1 mvneclipse 0m 14s branch-2.7 passed +1 findbugs 2m 57s branch-2.7 passed +1 javadoc 0m 59s branch-2.7 passed with JDK v1.8.0_131 +1 javadoc 1m 45s branch-2.7 passed with JDK v1.7.0_131 +1 mvninstall 0m 53s the patch passed +1 compile 0m 59s the patch passed with JDK v1.8.0_131 +1 javac 0m 59s the patch passed +1 compile 1m 0s the patch passed with JDK v1.7.0_131 +1 javac 1m 0s the patch passed -0 checkstyle 0m 27s hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 561 unchanged - 1 fixed = 562 total (was 562) +1 mvnsite 0m 57s the patch passed +1 mvneclipse 0m 12s the patch passed -1 whitespace 0m 0s The patch has 3522 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply -1 whitespace 1m 27s The patch 70 line(s) with tabs. +1 findbugs 3m 13s the patch passed +1 javadoc 0m 56s the patch passed with JDK v1.8.0_131 +1 javadoc 1m 43s the patch passed with JDK v1.7.0_131 -1 unit 44m 11s hadoop-hdfs in the patch failed with JDK v1.7.0_131. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 132m 46s Reason Tests JDK v1.8.0_131 Failed junit tests hadoop.hdfs.web.TestHttpsFileSystem   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots JDK v1.7.0_131 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots Subsystem Report/Notes Docker Image:yetus/hadoop:c420dfe JIRA Issue HDFS-11445 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12869243/HDFS-11445-branch-2.7-002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 2cc707c48a62 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2.7 / 8a0e5e3 Default Java 1.7.0_131 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_131 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_131 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/19541/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/19541/artifact/patchprocess/whitespace-eol.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/19541/artifact/patchprocess/whitespace-tabs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/19541/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_131.txt JDK v1.7.0_131 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/19541/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19541/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        vinayrpet Vinayakumar B added a comment -

        +1, latest patch looks good to me as well.

        Show
        vinayrpet Vinayakumar B added a comment - +1, latest patch looks good to me as well.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11782 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11782/)
        HDFS-11445. FSCK shows overall health stauts as corrupt even one replica (brahma: rev 2e41f8803dd46d1bab16c1b206c71be72ea260a1)

        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockInfo.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11782 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11782/ ) HDFS-11445 . FSCK shows overall health stauts as corrupt even one replica (brahma: rev 2e41f8803dd46d1bab16c1b206c71be72ea260a1) (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockInfo.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Committed to trunk,branch-2,branch-2.8,branch-2.8.1 and branch-2.7. Jing Zhao thanks a lot for great review..Vinayakumar B thanks for additional review.

        Show
        brahmareddy Brahma Reddy Battula added a comment - Committed to trunk , branch-2 , branch-2.8 , branch-2.8.1 and branch-2.7 . Jing Zhao thanks a lot for great review.. Vinayakumar B thanks for additional review.
        Hide
        vinodkv Vinod Kumar Vavilapalli added a comment -

        2.8.1 became a security release. Moving fix-version to 2.8.2 after the fact.

        Show
        vinodkv Vinod Kumar Vavilapalli added a comment - 2.8.1 became a security release. Moving fix-version to 2.8.2 after the fact.

          People

          • Assignee:
            brahmareddy Brahma Reddy Battula
            Reporter:
            brahmareddy Brahma Reddy Battula
          • Votes:
            0 Vote for this issue
            Watchers:
            14 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development