Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2978

The NameNode should expose name dir statuses via JMX

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0, 1.0.0
    • Fix Version/s: 0.23.3, 2.0.2-alpha
    • Component/s: namenode
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      We currently display this info on the NN web UI, so users who wish to monitor this must either do it manually or parse HTML. We should publish this information via JMX.

      1. HDFS-2978-branch-1.patch
        6 kB
        Aaron T. Myers
      2. HDFS-2978.patch
        7 kB
        Aaron T. Myers
      3. HDFS-2978.patch
        7 kB
        Aaron T. Myers

        Activity

        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #309 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/309/)
        svn merge -c 1293707 FIXES: HDFS-2978. The NameNode should expose name dir statuses via JMX. Contributed by Aaron T. Myers. (Revision 1359195)

        Result = SUCCESS
        daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359195
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #309 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/309/ ) svn merge -c 1293707 FIXES: HDFS-2978 . The NameNode should expose name dir statuses via JMX. Contributed by Aaron T. Myers. (Revision 1359195) Result = SUCCESS daryn : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359195 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
        Hide
        Tsz Wo Nicholas Sze added a comment -

        Merged to branch-1.0.

        Show
        Tsz Wo Nicholas Sze added a comment - Merged to branch-1.0.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1002 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1002/)
        HDFS-2978. The NameNode should expose name dir statuses via JMX. Contributed by Aaron T. Myers. (Revision 1293707)

        Result = SUCCESS
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1293707
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1002 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1002/ ) HDFS-2978 . The NameNode should expose name dir statuses via JMX. Contributed by Aaron T. Myers. (Revision 1293707) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1293707 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Build #208 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/208/)
        HDFS-2978. The NameNode should expose name dir statuses via JMX. Contributed by Aaron T. Myers. (Revision 1293709)

        Result = FAILURE
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1293709
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #208 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/208/ ) HDFS-2978 . The NameNode should expose name dir statuses via JMX. Contributed by Aaron T. Myers. (Revision 1293709) Result = FAILURE atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1293709 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #180 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/180/)
        HDFS-2978. The NameNode should expose name dir statuses via JMX. Contributed by Aaron T. Myers. (Revision 1293709)

        Result = SUCCESS
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1293709
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #180 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/180/ ) HDFS-2978 . The NameNode should expose name dir statuses via JMX. Contributed by Aaron T. Myers. (Revision 1293709) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1293709 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #967 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/967/)
        HDFS-2978. The NameNode should expose name dir statuses via JMX. Contributed by Aaron T. Myers. (Revision 1293707)

        Result = SUCCESS
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1293707
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #967 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/967/ ) HDFS-2978 . The NameNode should expose name dir statuses via JMX. Contributed by Aaron T. Myers. (Revision 1293707) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1293707 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12516063/HDFS-2978.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1911//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1911//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12516063/HDFS-2978.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1911//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1911//console This message is automatically generated.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Commit #597 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/597/)
        HDFS-2978. The NameNode should expose name dir statuses via JMX. Contributed by Aaron T. Myers. (Revision 1293709)

        Result = ABORTED
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1293709
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #597 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/597/ ) HDFS-2978 . The NameNode should expose name dir statuses via JMX. Contributed by Aaron T. Myers. (Revision 1293709) Result = ABORTED atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1293709 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #1792 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1792/)
        HDFS-2978. The NameNode should expose name dir statuses via JMX. Contributed by Aaron T. Myers. (Revision 1293707)

        Result = ABORTED
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1293707
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1792 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1792/ ) HDFS-2978 . The NameNode should expose name dir statuses via JMX. Contributed by Aaron T. Myers. (Revision 1293707) Result = ABORTED atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1293707 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
        Hide
        Aaron T. Myers added a comment -

        I've just committed this to branch-1, branch-0.23, and trunk.

        Thanks a lot for the review, Suresh.

        Show
        Aaron T. Myers added a comment - I've just committed this to branch-1, branch-0.23, and trunk. Thanks a lot for the review, Suresh.
        Hide
        Aaron T. Myers added a comment -

        Here's the patch I just committed to branch-1.

        Show
        Aaron T. Myers added a comment - Here's the patch I just committed to branch-1.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Commit #584 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/584/)
        HDFS-2978. The NameNode should expose name dir statuses via JMX. Contributed by Aaron T. Myers. (Revision 1293709)

        Result = SUCCESS
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1293709
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #584 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/584/ ) HDFS-2978 . The NameNode should expose name dir statuses via JMX. Contributed by Aaron T. Myers. (Revision 1293709) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1293709 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #1855 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1855/)
        HDFS-2978. The NameNode should expose name dir statuses via JMX. Contributed by Aaron T. Myers. (Revision 1293707)

        Result = SUCCESS
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1293707
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1855 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1855/ ) HDFS-2978 . The NameNode should expose name dir statuses via JMX. Contributed by Aaron T. Myers. (Revision 1293707) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1293707 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #1781 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1781/)
        HDFS-2978. The NameNode should expose name dir statuses via JMX. Contributed by Aaron T. Myers. (Revision 1293707)

        Result = SUCCESS
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1293707
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1781 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1781/ ) HDFS-2978 . The NameNode should expose name dir statuses via JMX. Contributed by Aaron T. Myers. (Revision 1293707) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1293707 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-0.23-Commit #596 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/596/)
        HDFS-2978. The NameNode should expose name dir statuses via JMX. Contributed by Aaron T. Myers. (Revision 1293709)

        Result = SUCCESS
        atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1293709
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java
        • /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #596 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/596/ ) HDFS-2978 . The NameNode should expose name dir statuses via JMX. Contributed by Aaron T. Myers. (Revision 1293709) Result = SUCCESS atm : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1293709 Files : /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java /hadoop/common/branches/branch-0.23/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMXBean.java
        Hide
        Aaron T. Myers added a comment -

        Here's a patch which addresses Suresh's comment. I'm going to commit this shortly

        Show
        Aaron T. Myers added a comment - Here's a patch which addresses Suresh's comment. I'm going to commit this shortly
        Hide
        Aaron T. Myers added a comment -

        @Aaron, the current patch should be fine as all the data currently exposed follows the same json serialize approach. However, at some point, it would be good to look at exactly how this is being used or meant to be used and fix the mbean output if needed.

        Sure, makes sense. Want to file a JIRA for that?

        Show
        Aaron T. Myers added a comment - @Aaron, the current patch should be fine as all the data currently exposed follows the same json serialize approach. However, at some point, it would be good to look at exactly how this is being used or meant to be used and fix the mbean output if needed. Sure, makes sense. Want to file a JIRA for that?
        Hide
        Hitesh Shah added a comment -

        @Aaron, the current patch should be fine as all the data currently exposed follows the same json serialize approach. However, at some point, it would be good to look at exactly how this is being used or meant to be used and fix the mbean output if needed.

        Also to clarify, if the data is being read via the /jmx http endpoint, it will return a valid json object ( mapped from the object being returned to the MX bean interface ) but I am not sure how this will affect scripts/programs that access the jmx data via a local connection in terms of whether they need to be tightly bound to the object type being returned as compared to be being a more flexible in handling a serialized json string.

        Show
        Hitesh Shah added a comment - @Aaron, the current patch should be fine as all the data currently exposed follows the same json serialize approach. However, at some point, it would be good to look at exactly how this is being used or meant to be used and fix the mbean output if needed. Also to clarify, if the data is being read via the /jmx http endpoint, it will return a valid json object ( mapped from the object being returned to the MX bean interface ) but I am not sure how this will affect scripts/programs that access the jmx data via a local connection in terms of whether they need to be tightly bound to the object type being returned as compared to be being a more flexible in handling a serialized json string.
        Hide
        Aaron T. Myers added a comment -

        Also, ignoring the above comment, would it be a good idea to have this available in 1.x too?

        Sure, good thinking. I'd be happy to provide such a patch.

        Todd/Hitesh - does Suresh's reply re: why JSON for JMX data address your concerns?

        Show
        Aaron T. Myers added a comment - Also, ignoring the above comment, would it be a good idea to have this available in 1.x too? Sure, good thinking. I'd be happy to provide such a patch. Todd/Hitesh - does Suresh's reply re: why JSON for JMX data address your concerns?
        Hide
        Suresh Srinivas added a comment -

        Exposing Strings and primitives is straight forward in JMX. But not so for more complicated data structures to non java programs. These interfaces were intended as a replacement of screen scraping of Namenode web UI, to be used in scripts etc. using local JMX connection. For such usages, the information is exposed java independent way.

        Show
        Suresh Srinivas added a comment - Exposing Strings and primitives is straight forward in JMX. But not so for more complicated data structures to non java programs. These interfaces were intended as a replacement of screen scraping of Namenode web UI, to be used in scripts etc. using local JMX connection. For such usages, the information is exposed java independent way.
        Hide
        Todd Lipcon added a comment -

        I agree with Hitesh - JMX is supposed to support arbitrarily nested map structures of java primitives, isn't it? I don't know it well, but I think we do this in other places.

        Show
        Todd Lipcon added a comment - I agree with Hitesh - JMX is supposed to support arbitrarily nested map structures of java primitives, isn't it? I don't know it well, but I think we do this in other places.
        Hide
        Hitesh Shah added a comment -

        @Aaron, Minor question on the use of emitting the data as a json-encoded string - any reason why this is being done for all the information exposed through the mbeans instead of using defined structures which are recognized and handled by the JMX interface in any case?

        Also, ignoring the above comment, would it be a good idea to have this available in 1.x too?

        Show
        Hitesh Shah added a comment - @Aaron, Minor question on the use of emitting the data as a json-encoded string - any reason why this is being done for all the information exposed through the mbeans instead of using defined structures which are recognized and handled by the JMX interface in any case? Also, ignoring the above comment, would it be a good idea to have this available in 1.x too?
        Hide
        Suresh Srinivas added a comment -

        Minor comment - In NameNodeMXBean.java can you please elaborate on what "name dirs" mean in the javadoc.
        +1 for the patch.

        Show
        Suresh Srinivas added a comment - Minor comment - In NameNodeMXBean.java can you please elaborate on what "name dirs" mean in the javadoc. +1 for the patch.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12515947/HDFS-2978.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1904//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1904//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12515947/HDFS-2978.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1904//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1904//console This message is automatically generated.
        Hide
        Aaron T. Myers added a comment -

        Here's a patch for trunk which addresses the issue.

        Show
        Aaron T. Myers added a comment - Here's a patch for trunk which addresses the issue.
        Hide
        Aaron T. Myers added a comment -

        One of the things that was done in 0.23 (available in 1.0) was to add every thing exposed in Namenode webUI to the JMX interfaces on the namenode. This interface is accessible over http. I believe this information was added to NN WebUI, post that. We should add it to JMX interfaces.

        Cool, so you agree that this info is not currently present? Just making sure.

        One thing I notice with some of these Jiras is, Aaron you keep saying metrics. Metrics to me is an interface where keep track of peg counts, rate etc, for measurement purposes. The information such as dirs etc to me belongs to management interface and that is where JMX interfaces come in. So we should add this to the JMX interface. Also we need to enforce adding any info added to NN web UI to the JMX interface.

        You're right - I've been inaccurate when using the term "metrics" for this. My apologies.

        Show
        Aaron T. Myers added a comment - One of the things that was done in 0.23 (available in 1.0) was to add every thing exposed in Namenode webUI to the JMX interfaces on the namenode. This interface is accessible over http. I believe this information was added to NN WebUI, post that. We should add it to JMX interfaces. Cool, so you agree that this info is not currently present? Just making sure. One thing I notice with some of these Jiras is, Aaron you keep saying metrics. Metrics to me is an interface where keep track of peg counts, rate etc, for measurement purposes. The information such as dirs etc to me belongs to management interface and that is where JMX interfaces come in. So we should add this to the JMX interface. Also we need to enforce adding any info added to NN web UI to the JMX interface. You're right - I've been inaccurate when using the term "metrics" for this. My apologies.
        Hide
        Suresh Srinivas added a comment -

        One of the things that was done in 0.23 (available in 1.0) was to add every thing exposed in Namenode webUI to the JMX interfaces on the namenode. This interface is accessible over http. I believe this information was added to NN WebUI, post that. We should add it to JMX interfaces.

        One thing I notice with some of these Jiras is, Aaron you keep saying metrics. Metrics to me is an interface where keep track of peg counts, rate etc, for measurement purposes. The information such as dirs etc to me belongs to management interface and that is where JMX interfaces come in. So we should add this to the JMX interface. Also we need to enforce adding any info added to NN web UI to the JMX interface.

        Show
        Suresh Srinivas added a comment - One of the things that was done in 0.23 (available in 1.0) was to add every thing exposed in Namenode webUI to the JMX interfaces on the namenode. This interface is accessible over http. I believe this information was added to NN WebUI, post that. We should add it to JMX interfaces. One thing I notice with some of these Jiras is, Aaron you keep saying metrics. Metrics to me is an interface where keep track of peg counts, rate etc, for measurement purposes. The information such as dirs etc to me belongs to management interface and that is where JMX interfaces come in. So we should add this to the JMX interface. Also we need to enforce adding any info added to NN web UI to the JMX interface.
        Hide
        Aaron T. Myers added a comment -

        Hey Suresh, where? I don't see anything which looks like what this JIRA is describing at "/jmx" on the NN.

        Show
        Aaron T. Myers added a comment - Hey Suresh, where? I don't see anything which looks like what this JIRA is describing at "/jmx" on the NN.
        Hide
        Suresh Srinivas added a comment -

        There is already JMX and HTTP based access to JMX interfaces available for getting this information. Most of the things in NN web UI is available through these interfaces.

        Show
        Suresh Srinivas added a comment - There is already JMX and HTTP based access to JMX interfaces available for getting this information. Most of the things in NN web UI is available through these interfaces.

          People

          • Assignee:
            Aaron T. Myers
            Reporter:
            Aaron T. Myers
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development