Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8031 Follow-on work for erasure coding phase I (striping layout)
  3. HDFS-10827

When there are unrecoverable ec block groups, Namenode Web UI shows "There are X missing blocks." but doesn't show the block names.

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha2
    • Component/s: erasure-coding
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      For RS-6-3, when there is one ec block group and
      1) 0~3 out of 9 internal blocks are missing, NN Web UI doesn't show any warns.
      2) 4~8 out of 9 internal blocks are missing, NN Web UI shows "There are 1 missing blocks." but doesn't show the block names. (please see case_2.png)
      3) 9 out of 9 internal blocks are missing, NN Web UI shows "There are 1 missing blocks." and also shows the block name. (please see case_3.png)

      We should fix the case 2). I think NN Web UI should show the block names since the ec block group is unrecoverable.

      The values come from JMX. "There are X missing blocks." is NumberOfMissingBlocks and the block names are CorruptFiles.

      1. case_2.png
        46 kB
        Takanobu Asanuma
      2. case_3.png
        52 kB
        Takanobu Asanuma
      3. HDFS-10827.1.patch
        8 kB
        Takanobu Asanuma
      4. HDFS-10827.2.patch
        9 kB
        Takanobu Asanuma
      5. HDFS-10827.3.patch
        8 kB
        Takanobu Asanuma
      6. HDFS-10827.4.patch
        7 kB
        Takanobu Asanuma

        Issue Links

          Activity

          Hide
          shahrs87 Rushabh S Shah added a comment -

          We don't use ec in our builds but we see the error as shown in case_2.png but we were not able to track down the root cause.

          Show
          shahrs87 Rushabh S Shah added a comment - We don't use ec in our builds but we see the error as shown in case_2.png but we were not able to track down the root cause.
          Hide
          tasanuma0829 Takanobu Asanuma added a comment -

          Thank you for the information. Although we haven't seen this problem for replication blocks, I will keep in mind it.

          Show
          tasanuma0829 Takanobu Asanuma added a comment - Thank you for the information. Although we haven't seen this problem for replication blocks, I will keep in mind it.
          Hide
          tasanuma0829 Takanobu Asanuma added a comment -

          I uploaded a new patch for the problem of EC block groups.

          Show
          tasanuma0829 Takanobu Asanuma added a comment - I uploaded a new patch for the problem of EC block groups.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 17s trunk passed
          +1 compile 0m 52s trunk passed
          +1 checkstyle 0m 29s trunk passed
          +1 mvnsite 0m 55s trunk passed
          +1 mvneclipse 0m 15s trunk passed
          +1 findbugs 1m 54s trunk passed
          +1 javadoc 0m 57s trunk passed
          +1 mvninstall 0m 59s the patch passed
          +1 compile 0m 52s the patch passed
          +1 javac 0m 52s the patch passed
          +1 checkstyle 0m 26s the patch passed
          +1 mvnsite 0m 58s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 2m 0s the patch passed
          +1 javadoc 0m 57s the patch passed
          -1 unit 61m 55s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          82m 52s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.namenode.TestAddStripedBlockInFBR



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HDFS-10827
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830474/HDFS-10827.1.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 10727b9a923f 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / e17a497
          Default Java 1.8.0_101
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16879/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16879/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16879/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 17s trunk passed +1 compile 0m 52s trunk passed +1 checkstyle 0m 29s trunk passed +1 mvnsite 0m 55s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 54s trunk passed +1 javadoc 0m 57s trunk passed +1 mvninstall 0m 59s the patch passed +1 compile 0m 52s the patch passed +1 javac 0m 52s the patch passed +1 checkstyle 0m 26s the patch passed +1 mvnsite 0m 58s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 0s the patch passed +1 javadoc 0m 57s the patch passed -1 unit 61m 55s hadoop-hdfs in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 82m 52s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestAddStripedBlockInFBR Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10827 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12830474/HDFS-10827.1.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 10727b9a923f 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e17a497 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16879/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16879/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16879/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          tasanuma0829 Takanobu Asanuma added a comment -

          The failed test is passed in my local computer.

          Show
          tasanuma0829 Takanobu Asanuma added a comment - The failed test is passed in my local computer.
          Hide
          tasanuma0829 Takanobu Asanuma added a comment -

          I updated the patch with some improvements based on the discussion of HDFS-10826.

          Show
          tasanuma0829 Takanobu Asanuma added a comment - I updated the patch with some improvements based on the discussion of HDFS-10826 .
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 6m 58s trunk passed
          +1 compile 0m 44s trunk passed
          +1 checkstyle 0m 28s trunk passed
          +1 mvnsite 0m 51s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 40s trunk passed
          +1 javadoc 0m 55s trunk passed
          +1 mvninstall 0m 45s the patch passed
          +1 compile 0m 42s the patch passed
          +1 javac 0m 42s the patch passed
          +1 checkstyle 0m 25s the patch passed
          +1 mvnsite 0m 47s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 46s the patch passed
          +1 javadoc 0m 52s the patch passed
          -1 unit 58m 53s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 21s The patch does not generate ASF License warnings.
          77m 59s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.namenode.TestDecommissioningStatus



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HDFS-10827
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12831262/HDFS-10827.2.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 17a7e7d4d59f 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / fe9ebe2
          Default Java 1.8.0_101
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/16971/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16971/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16971/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 58s trunk passed +1 compile 0m 44s trunk passed +1 checkstyle 0m 28s trunk passed +1 mvnsite 0m 51s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 40s trunk passed +1 javadoc 0m 55s trunk passed +1 mvninstall 0m 45s the patch passed +1 compile 0m 42s the patch passed +1 javac 0m 42s the patch passed +1 checkstyle 0m 25s the patch passed +1 mvnsite 0m 47s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 46s the patch passed +1 javadoc 0m 52s the patch passed -1 unit 58m 53s hadoop-hdfs in the patch failed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 77m 59s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestDecommissioningStatus Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10827 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12831262/HDFS-10827.2.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 17a7e7d4d59f 3.13.0-92-generic #139-Ubuntu SMP Tue Jun 28 20:42:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / fe9ebe2 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16971/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16971/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16971/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          tasanuma0829 Takanobu Asanuma added a comment -

          The failed test can pass in my local computer.

          Show
          tasanuma0829 Takanobu Asanuma added a comment - The failed test can pass in my local computer.
          Hide
          jingzhao Jing Zhao added a comment -

          Thanks for working on this, Takanobu Asanuma! The patch looks good to me overall. Some minors:

          1. I think we can rename "isCorrupt" to "isMissing". In the outer "while" loop, we're scanning all the blocks with QUEUE_WITH_CORRUPT_BLOCKS priority, and this "if" logic is to select missing blocks.
          2. We may also want to take into account the decommissioning/decommissioned internal blocks for the check.
          3. As follow-on work, maybe we can create a utility function to check if a block is corrupted/missing, considering this is widely used in BlockManager/FSNamesystem.
          4. For the unit test, do you think we can use MiniDFSCluster#corruptReplica to simplify the code?
          Show
          jingzhao Jing Zhao added a comment - Thanks for working on this, Takanobu Asanuma ! The patch looks good to me overall. Some minors: I think we can rename "isCorrupt" to "isMissing". In the outer "while" loop, we're scanning all the blocks with QUEUE_WITH_CORRUPT_BLOCKS priority, and this "if" logic is to select missing blocks. We may also want to take into account the decommissioning/decommissioned internal blocks for the check. As follow-on work, maybe we can create a utility function to check if a block is corrupted/missing, considering this is widely used in BlockManager/FSNamesystem. For the unit test, do you think we can use MiniDFSCluster#corruptReplica to simplify the code?
          Hide
          tasanuma0829 Takanobu Asanuma added a comment - - edited

          Thank you for your kind review, Jing Zhao!

          On second thought, we may not need to check it, like below. Because the blocks in priorityQueues.get(QUEUE_WITH_CORRUPT_BLOCKS) is already took into account various cases(replication/ec, decommissioning/decommissioned) in LowRedundancyBlocks.getPriority. What do you think?

          FSNameSystem.listCorruptFileBlocks
          while (blkIterator.hasNext()) {
            BlockInfo blk = blkIterator.next();
            final INodeFile inode = getBlockCollection(blk);
            skip++;
            if (inode != null) {
              ...
            }
          }
          
          Show
          tasanuma0829 Takanobu Asanuma added a comment - - edited Thank you for your kind review, Jing Zhao ! On second thought, we may not need to check it, like below. Because the blocks in priorityQueues.get(QUEUE_WITH_CORRUPT_BLOCKS) is already took into account various cases(replication/ec, decommissioning/decommissioned) in LowRedundancyBlocks.getPriority . What do you think? FSNameSystem.listCorruptFileBlocks while (blkIterator.hasNext()) { BlockInfo blk = blkIterator.next(); final INodeFile inode = getBlockCollection(blk); skip++; if (inode != null ) { ... } }
          Hide
          tasanuma0829 Takanobu Asanuma added a comment -

          Jing Zhao
          Based on my last comment, I updated the patch to see the test results. It doesn't include all your advice yet.

          Show
          tasanuma0829 Takanobu Asanuma added a comment - Jing Zhao Based on my last comment, I updated the patch to see the test results. It doesn't include all your advice yet.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 49s trunk passed
          +1 compile 0m 49s trunk passed
          +1 checkstyle 0m 30s trunk passed
          +1 mvnsite 0m 54s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 52s trunk passed
          +1 javadoc 0m 39s trunk passed
          +1 mvninstall 0m 46s the patch passed
          +1 compile 0m 43s the patch passed
          +1 javac 0m 43s the patch passed
          +1 checkstyle 0m 27s the patch passed
          +1 mvnsite 0m 52s the patch passed
          +1 mvneclipse 0m 11s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 2m 6s the patch passed
          +1 javadoc 0m 37s the patch passed
          +1 unit 58m 55s hadoop-hdfs in the patch passed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          79m 8s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HDFS-10827
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12832857/HDFS-10827.3.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux dad7290f7f85 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 6476934
          Default Java 1.8.0_101
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17119/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17119/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 49s trunk passed +1 compile 0m 49s trunk passed +1 checkstyle 0m 30s trunk passed +1 mvnsite 0m 54s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 52s trunk passed +1 javadoc 0m 39s trunk passed +1 mvninstall 0m 46s the patch passed +1 compile 0m 43s the patch passed +1 javac 0m 43s the patch passed +1 checkstyle 0m 27s the patch passed +1 mvnsite 0m 52s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 6s the patch passed +1 javadoc 0m 37s the patch passed +1 unit 58m 55s hadoop-hdfs in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 79m 8s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10827 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12832857/HDFS-10827.3.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux dad7290f7f85 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6476934 Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17119/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17119/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          jingzhao Jing Zhao added a comment -

          Takanobu Asanuma, I think you're right. We actually do not need to add that extra check there.

          Show
          jingzhao Jing Zhao added a comment - Takanobu Asanuma , I think you're right. We actually do not need to add that extra check there.
          Hide
          tasanuma0829 Takanobu Asanuma added a comment -

          Thanks for your confirmation. I will upload the next patch soon.

          Show
          tasanuma0829 Takanobu Asanuma added a comment - Thanks for your confirmation. I will upload the next patch soon.
          Hide
          tasanuma0829 Takanobu Asanuma added a comment -

          This jira will solve HDFS-10975.

          Show
          tasanuma0829 Takanobu Asanuma added a comment - This jira will solve HDFS-10975 .
          Hide
          tasanuma0829 Takanobu Asanuma added a comment -

          I uploaded a new patch. It uses MiniDFSCluster#corruptReplica.

          Show
          tasanuma0829 Takanobu Asanuma added a comment - I uploaded a new patch. It uses MiniDFSCluster#corruptReplica .
          Hide
          tasanuma0829 Takanobu Asanuma added a comment -

          As follow-on work, maybe we can create a utility function to check if a block is corrupted/missing, considering this is widely used in BlockManager/FSNamesystem.

          I agree with you. It is also useful for others. For example, fsck counts some low-redundancy metrics in its own way at the moment. It would be good if it uses the same logic with LowRedundancyBlocks. I will consider it more and create a new jira.

          Show
          tasanuma0829 Takanobu Asanuma added a comment - As follow-on work, maybe we can create a utility function to check if a block is corrupted/missing, considering this is widely used in BlockManager/FSNamesystem. I agree with you. It is also useful for others. For example, fsck counts some low-redundancy metrics in its own way at the moment. It would be good if it uses the same logic with LowRedundancyBlocks . I will consider it more and create a new jira.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 12m 47s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 35s trunk passed
          +1 compile 0m 43s trunk passed
          +1 checkstyle 0m 28s trunk passed
          +1 mvnsite 0m 51s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 41s trunk passed
          +1 javadoc 0m 40s trunk passed
          +1 mvninstall 0m 47s the patch passed
          +1 compile 0m 45s the patch passed
          +1 javac 0m 45s the patch passed
          +1 checkstyle 0m 26s the patch passed
          +1 mvnsite 0m 51s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 50s the patch passed
          +1 javadoc 0m 35s the patch passed
          -1 unit 62m 25s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 19s The patch does not generate ASF License warnings.
          94m 24s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestAppendSnapshotTruncate



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HDFS-10827
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833293/HDFS-10827.4.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux e95e5c016e87 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / cf3f43e
          Default Java 1.8.0_101
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/17148/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17148/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17148/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 12m 47s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 35s trunk passed +1 compile 0m 43s trunk passed +1 checkstyle 0m 28s trunk passed +1 mvnsite 0m 51s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 41s trunk passed +1 javadoc 0m 40s trunk passed +1 mvninstall 0m 47s the patch passed +1 compile 0m 45s the patch passed +1 javac 0m 45s the patch passed +1 checkstyle 0m 26s the patch passed +1 mvnsite 0m 51s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 50s the patch passed +1 javadoc 0m 35s the patch passed -1 unit 62m 25s hadoop-hdfs in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 94m 24s Reason Tests Failed junit tests hadoop.hdfs.TestAppendSnapshotTruncate Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10827 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12833293/HDFS-10827.4.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e95e5c016e87 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / cf3f43e Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/17148/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17148/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17148/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          jingzhao Jing Zhao added a comment -

          +1 on the latest patch. I've committed it into trunk. Thanks for the contribution, Takanobu Asanuma!

          Show
          jingzhao Jing Zhao added a comment - +1 on the latest patch. I've committed it into trunk. Thanks for the contribution, Takanobu Asanuma !
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10614 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10614/)
          HDFS-10827. When there are unrecoverable ec block groups, Namenode Web (jing9: rev adb96e109f1ab4a2c3d469e716c084d0a891b951)

          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10614 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10614/ ) HDFS-10827 . When there are unrecoverable ec block groups, Namenode Web (jing9: rev adb96e109f1ab4a2c3d469e716c084d0a891b951) (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
          Hide
          tasanuma0829 Takanobu Asanuma added a comment -

          Jing Zhao
          Thank you for reviewing and committing!

          Rushabh S Shah
          Although I couldn't reproduce the same problem for replication, this change may fix it.

          Show
          tasanuma0829 Takanobu Asanuma added a comment - Jing Zhao Thank you for reviewing and committing! Rushabh S Shah Although I couldn't reproduce the same problem for replication, this change may fix it.

            People

            • Assignee:
              tasanuma0829 Takanobu Asanuma
              Reporter:
              tasanuma0829 Takanobu Asanuma
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development