Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-850

Display more memory details on the web ui

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Changes the format of the message with Heap usage on the NameNode web page.

      Description

      With the HDFS-94 being commited, the namenode will use JMX memory beans to get information about heap usage.
      They provide us with additional information such as NonHeap memory usage and Heap Commited and Initialized memory in addition to Used and Max.
      It will be useful to see that additional information on the NameNode web ui.

      1. HDFS-850.patch
        2 kB
        Dmytro Molkov
      2. screenshot-2.jpg
        65 kB
        Dmytro Molkov
      3. HDFS-850.patch
        2 kB
        Dmytro Molkov
      4. screenshot-1.jpg
        60 kB
        Dmytro Molkov
      5. HDFS-850.patch
        2 kB
        Dmytro Molkov

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #275 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk/275/)

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #275 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk/275/ )
          Hide
          Hudson added a comment -

          Integrated in Hdfs-Patch-h5.grid.sp2.yahoo.net #302 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/302/)

          Show
          Hudson added a comment - Integrated in Hdfs-Patch-h5.grid.sp2.yahoo.net #302 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/302/ )
          Hide
          Hudson added a comment -

          Integrated in Hdfs-Patch-h2.grid.sp2.yahoo.net #146 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/146/)

          Show
          Hudson added a comment - Integrated in Hdfs-Patch-h2.grid.sp2.yahoo.net #146 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/146/ )
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #213 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/213/)
          . The WebUI displays more details about namenode memory usage.
          (Dmytro Molkov via dhruba)

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #213 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/213/ ) . The WebUI displays more details about namenode memory usage. (Dmytro Molkov via dhruba)
          Hide
          dhruba borthakur added a comment -

          I just committed this. Thanks Dmytro!

          Show
          dhruba borthakur added a comment - I just committed this. Thanks Dmytro!
          Hide
          dhruba borthakur added a comment -

          +1 Code looks good. I will commit it.

          Show
          dhruba borthakur added a comment - +1 Code looks good. I will commit it.
          Hide
          Hemanth Yamijala added a comment -

          This same code path should be added to the JobTracker. Is there an equivalent JIRA for it?

          I filed MAPREDUCE-1396 for this.

          Show
          Hemanth Yamijala added a comment - This same code path should be added to the JobTracker. Is there an equivalent JIRA for it? I filed MAPREDUCE-1396 for this.
          Hide
          Suresh Srinivas added a comment -

          This patch does not apply to the trunk. Looks like there are conflicting changes. Could you please upload a new patch.

          Show
          Suresh Srinivas added a comment - This patch does not apply to the trunk. Looks like there are conflicting changes. Could you please upload a new patch.
          Hide
          Dmytro Molkov added a comment -

          This is a UI change so no test included with it

          Show
          Dmytro Molkov added a comment - This is a UI change so no test included with it
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12430071/HDFS-850.patch
          against trunk revision 898573.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/186/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/186/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/186/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/186/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12430071/HDFS-850.patch against trunk revision 898573. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/186/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/186/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/186/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/186/console This message is automatically generated.
          Hide
          Dmytro Molkov added a comment -

          Resubmitting the patch, because the system tried to apply the screenshot

          Show
          Dmytro Molkov added a comment - Resubmitting the patch, because the system tried to apply the screenshot
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12429990/screenshot-2.jpg
          against trunk revision 898573.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 patch. The patch command could not apply the patch.

          Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/185/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12429990/screenshot-2.jpg against trunk revision 898573. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/185/console This message is automatically generated.
          Hide
          Allen Wittenauer added a comment -

          This same code path should be added to the JobTracker. Is there an equivalent JIRA for it?

          Show
          Allen Wittenauer added a comment - This same code path should be added to the JobTracker. Is there an equivalent JIRA for it?
          Hide
          Dmytro Molkov added a comment -

          Updated view

          Show
          Dmytro Molkov added a comment - Updated view
          Hide
          Dmytro Molkov added a comment -

          Updated patch addressing comments by Suresh

          Show
          Dmytro Molkov added a comment - Updated patch addressing comments by Suresh
          Hide
          Suresh Srinivas added a comment -

          This information will be useful.

          1. Can you print the memory usage in separate lines from files and block counts.
          2. Current format is: Heap Size <used> / <max>. Committed Heap is <committed>. Using the following format would better explain the information: Heap memory used <used> is <%> of committed memory <commited>. Maximum heap memory is <max>. This also calculates the % usage from the committed memory (which is what is currently available) instead of maximum size the process can grow to.
          Show
          Suresh Srinivas added a comment - This information will be useful. Can you print the memory usage in separate lines from files and block counts. Current format is: Heap Size <used> / <max>. Committed Heap is <committed>. Using the following format would better explain the information: Heap memory used <used> is <%> of committed memory <commited>. Maximum heap memory is <max>. This also calculates the % usage from the committed memory (which is what is currently available) instead of maximum size the process can grow to.
          Hide
          Dmytro Molkov added a comment -

          What it looks like.

          Show
          Dmytro Molkov added a comment - What it looks like.

            People

            • Assignee:
              Dmytro Molkov
              Reporter:
              Dmytro Molkov
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development