Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7166

SbNN Web UI shows #Under replicated blocks and #pending deletion blocks

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: ha
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      I believe that's an regression of HDFS-5333
      According to HDFS-2901 and HDFS-6178
      The Standby Namenode doesn't compute replication queues, we shouldn't show under-replicated/missing blocks or corrupt files.

      1. HDFS-7166.001.patch
        2 kB
        Wei-Chiu Chuang

        Issue Links

          Activity

          Hide
          wheat9 Haohui Mai added a comment -

          Does the information show up on the new Web UI? Just checking the code, it looks like that the information no longer exists.

          Can you please provide a screenshot to better illustrate the issue?

          Show
          wheat9 Haohui Mai added a comment - Does the information show up on the new Web UI? Just checking the code, it looks like that the information no longer exists. Can you please provide a screenshot to better illustrate the issue?
          Hide
          jyu@cloudera.com Juan Yu added a comment -

          they are in hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html

          Show
          jyu@cloudera.com Juan Yu added a comment - they are in hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html
          Hide
          wheat9 Haohui Mai added a comment -

          My bad, I though you meant secondary NN. Thanks for the explanation.

          Show
          wheat9 Haohui Mai added a comment - My bad, I though you meant secondary NN. Thanks for the explanation.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Haohui Mai: I have an initial patch, but I'm not sure how to unit test a html/js code.
          Could you shed some lights on how you developed and test it? Thank you very much.

          Show
          jojochuang Wei-Chiu Chuang added a comment - Haohui Mai : I have an initial patch, but I'm not sure how to unit test a html/js code. Could you shed some lights on how you developed and test it? Thank you very much.
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Rev01:
          SbNN will not show corrupted blocks, under-replicated blocks, missing block. In addition, it should also not show blocks that are pending deletion, according to TestStandbyBlockManagement.

          I tested this patch on a CDH cluster with HA and works as expected.

          Show
          jojochuang Wei-Chiu Chuang added a comment - Rev01: SbNN will not show corrupted blocks, under-replicated blocks, missing block. In addition, it should also not show blocks that are pending deletion, according to TestStandbyBlockManagement . I tested this patch on a CDH cluster with HA and works as expected.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 23s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 asflicense 0m 28s Patch does not generate ASF License warnings.
          1m 11s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12791488/HDFS-7166.001.patch
          JIRA Issue HDFS-7166
          Optional Tests asflicense
          uname Linux ff0f744c6dc1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / cbd3132
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14718/console
          Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 23s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 whitespace 0m 0s Patch has no whitespace issues. +1 asflicense 0m 28s Patch does not generate ASF License warnings. 1m 11s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12791488/HDFS-7166.001.patch JIRA Issue HDFS-7166 Optional Tests asflicense uname Linux ff0f744c6dc1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / cbd3132 modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14718/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          wheat9 Haohui Mai added a comment -

          LGTM. +1.

          For unit tests I will file another jira to track it.

          Show
          wheat9 Haohui Mai added a comment - LGTM. +1. For unit tests I will file another jira to track it.
          Hide
          wheat9 Haohui Mai added a comment -

          Committed to trunk, branch-2 and branch-2.8. Thanks Wei-Chiu Chuang for the contributions.

          Show
          wheat9 Haohui Mai added a comment - Committed to trunk, branch-2 and branch-2.8. Thanks Wei-Chiu Chuang for the contributions.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9480 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9480/)
          HDFS-7166. SbNN Web UI shows #Under replicated blocks and #pending (wheat9: rev 8a3f0cb25540c7e70471aebcdd408feb478f878e)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9480 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9480/ ) HDFS-7166 . SbNN Web UI shows #Under replicated blocks and #pending (wheat9: rev 8a3f0cb25540c7e70471aebcdd408feb478f878e) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Thanks Haohui Mai for reviewing and committing!

          Show
          jojochuang Wei-Chiu Chuang added a comment - Thanks Haohui Mai for reviewing and committing!

            People

            • Assignee:
              jojochuang Wei-Chiu Chuang
              Reporter:
              jyu@cloudera.com Juan Yu
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development