Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11247

Add a test to verify NameNodeMXBean#getDecomNodes() and Live/Dead Decom Nodes shown in NameNode WebUI

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha1
    • Fix Version/s: 3.0.0-alpha2
    • Component/s: None
    • Labels:
      None

      Description

      Add unit test to verify the following

      • NameNodeMXBean#getDecomNodes() – Decommission in progress nodes – Displayed under "Decommissioning" in NameNode WebUI page dfshealth.html#tab-datanode
      • Decommissioned Live and Dead nodes displayed under "Summary" in dfshealth.html#tab-overview page.
      1. HDFS-11247.01.patch
        4 kB
        Manoj Govindassamy
      2. HDFS-11247.02.patch
        5 kB
        Manoj Govindassamy

        Activity

        Hide
        manojg Manoj Govindassamy added a comment -

        Attached a patch to verify NameNodeMXBean#getDecomNodes() and other Decommissioning status used for the display in NameNide WebUI.
        Xiao Chen, can you please take a look a the patch ?

        Show
        manojg Manoj Govindassamy added a comment - Attached a patch to verify NameNodeMXBean#getDecomNodes() and other Decommissioning status used for the display in NameNide WebUI. Xiao Chen , can you please take a look a the patch ?
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 24s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 16s trunk passed
        +1 compile 0m 53s trunk passed
        +1 checkstyle 0m 28s trunk passed
        +1 mvnsite 0m 59s trunk passed
        +1 mvneclipse 0m 15s trunk passed
        +1 findbugs 1m 46s trunk passed
        +1 javadoc 0m 44s trunk passed
        +1 mvninstall 0m 52s the patch passed
        +1 compile 0m 45s the patch passed
        +1 javac 0m 45s the patch passed
        +1 checkstyle 0m 24s the patch passed
        +1 mvnsite 0m 52s the patch passed
        +1 mvneclipse 0m 11s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 2m 6s the patch passed
        +1 javadoc 0m 37s the patch passed
        -1 unit 95m 32s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 24s The patch does not generate ASF License warnings.
        115m 41s



        Reason Tests
        Failed junit tests hadoop.tracing.TestTracing
          hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11247
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843331/HDFS-11247.01.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 290368792a22 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 64a2d5b
        Default Java 1.8.0_111
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/17861/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17861/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17861/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 24s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 16s trunk passed +1 compile 0m 53s trunk passed +1 checkstyle 0m 28s trunk passed +1 mvnsite 0m 59s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 46s trunk passed +1 javadoc 0m 44s trunk passed +1 mvninstall 0m 52s the patch passed +1 compile 0m 45s the patch passed +1 javac 0m 45s the patch passed +1 checkstyle 0m 24s the patch passed +1 mvnsite 0m 52s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 6s the patch passed +1 javadoc 0m 37s the patch passed -1 unit 95m 32s hadoop-hdfs in the patch failed. +1 asflicense 0m 24s The patch does not generate ASF License warnings. 115m 41s Reason Tests Failed junit tests hadoop.tracing.TestTracing   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11247 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843331/HDFS-11247.01.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 290368792a22 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 64a2d5b Default Java 1.8.0_111 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/17861/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17861/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17861/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        manojg Manoj Govindassamy added a comment -

        Unit test failures doesn't look related to patch.

        Show
        manojg Manoj Govindassamy added a comment - Unit test failures doesn't look related to patch.
        Hide
        xiaochen Xiao Chen added a comment -

        Thanks Manoj for widening our test coverage.

        LGTM except 1 nit:

        • GenericTestUtils#waitFor is usually used to wait for a condition in test. I think the while loops for recheck and fsn.getNumDecomLiveDataNodes() != 1 could be replaced with that, to be more consistent with Hadoop code base.

        (There is also LambdaTestUtils to be used, but harder to backport to earlier versions.)

        Show
        xiaochen Xiao Chen added a comment - Thanks Manoj for widening our test coverage. LGTM except 1 nit: GenericTestUtils#waitFor is usually used to wait for a condition in test. I think the while loops for recheck and fsn.getNumDecomLiveDataNodes() != 1 could be replaced with that, to be more consistent with Hadoop code base. (There is also LambdaTestUtils to be used, but harder to backport to earlier versions.)
        Hide
        manojg Manoj Govindassamy added a comment -

        Thanks for the review Xiao Chen. Incorporated your review comments on using GenericTestUtils.waitFor(). Please take a look.

        Show
        manojg Manoj Govindassamy added a comment - Thanks for the review Xiao Chen . Incorporated your review comments on using GenericTestUtils.waitFor(). Please take a look.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 13m 26s trunk passed
        +1 compile 0m 50s trunk passed
        +1 checkstyle 0m 28s trunk passed
        +1 mvnsite 0m 54s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 2m 14s trunk passed
        +1 javadoc 0m 54s trunk passed
        +1 mvninstall 1m 0s the patch passed
        +1 compile 0m 57s the patch passed
        +1 javac 0m 57s the patch passed
        +1 checkstyle 0m 27s the patch passed
        +1 mvnsite 1m 3s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 2m 18s the patch passed
        +1 javadoc 0m 44s the patch passed
        -1 unit 76m 42s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 19s The patch does not generate ASF License warnings.
        104m 29s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeUUID
          hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA
          hadoop.hdfs.TestErasureCodeBenchmarkThroughput



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11247
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843991/HDFS-11247.02.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux f1b529085221 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 3583413
        Default Java 1.8.0_111
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/17908/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17908/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17908/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 13m 26s trunk passed +1 compile 0m 50s trunk passed +1 checkstyle 0m 28s trunk passed +1 mvnsite 0m 54s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 2m 14s trunk passed +1 javadoc 0m 54s trunk passed +1 mvninstall 1m 0s the patch passed +1 compile 0m 57s the patch passed +1 javac 0m 57s the patch passed +1 checkstyle 0m 27s the patch passed +1 mvnsite 1m 3s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 18s the patch passed +1 javadoc 0m 44s the patch passed -1 unit 76m 42s hadoop-hdfs in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 104m 29s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeUUID   hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA   hadoop.hdfs.TestErasureCodeBenchmarkThroughput Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11247 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12843991/HDFS-11247.02.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux f1b529085221 3.13.0-103-generic #150-Ubuntu SMP Thu Nov 24 10:34:17 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 3583413 Default Java 1.8.0_111 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/17908/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17908/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17908/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        manojg Manoj Govindassamy added a comment -

        Xiao Chen, test failures are not related to the patch v02.

        Show
        manojg Manoj Govindassamy added a comment - Xiao Chen , test failures are not related to the patch v02.
        Hide
        xiaochen Xiao Chen added a comment -

        Thanks for revving Manoj. +1 on patch 2, committing this.

        Show
        xiaochen Xiao Chen added a comment - Thanks for revving Manoj. +1 on patch 2, committing this.
        Hide
        xiaochen Xiao Chen added a comment -

        Committed to trunk, thanks Manoj for the contribution!

        Show
        xiaochen Xiao Chen added a comment - Committed to trunk, thanks Manoj for the contribution!
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11020 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11020/)
        HDFS-11247. Add a test to verify NameNodeMXBean#getDecomNodes() and (xiao: rev 4af66b1d664b05590c39e34ae04f1f304c3cd227)

        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11020 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11020/ ) HDFS-11247 . Add a test to verify NameNodeMXBean#getDecomNodes() and (xiao: rev 4af66b1d664b05590c39e34ae04f1f304c3cd227) (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java

          People

          • Assignee:
            manojg Manoj Govindassamy
            Reporter:
            manojg Manoj Govindassamy
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development