Hadoop Common
  1. Hadoop Common
  2. HADOOP-7144

Expose JMX with something like JMXProxyServlet

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.20.204.0, 0.23.0
    • Component/s: None
    • Labels:
    • Hadoop Flags:
      Reviewed
    • Tags:
      jmx, metrics, mbeans

      Description

      Much of the Hadoop metrics and status info is available via JMX, especially since 0.20.100, and 0.22+ (HDFS-1318, HADOOP-6728 etc.) For operations staff not familiar JMX setup, especially JMX with SSL and firewall tunnelling, the usage can be daunting. Using a JMXProxyServlet (a la Tomcat) to translate JMX attributes into JSON output would make a lot of non-Java admins happy.

      We could probably use Tomcat's JMXProxyServlet code directly, if it's already output some standard format (JSON or XML etc.) The code is simple enough to port over and can probably integrate with the common HttpServer as one of the default servelet (maybe /jmx) for the pluggable security.

      1. HADOOP-7411-trunk-V2.patch
        18 kB
        Robert Joseph Evans
      2. HADOOP-7411-0.20.20X-V2.patch
        18 kB
        Robert Joseph Evans
      3. jmx.json
        30 kB
        Robert Joseph Evans
      4. HADOOP-7411-trunk-V1.patch
        17 kB
        Robert Joseph Evans
      5. HADOOP-7411-0.20.20X-V1.patch
        17 kB
        Robert Joseph Evans
      6. HADOOP-7411-trunk-alpha.patch
        15 kB
        Robert Joseph Evans

        Issue Links

          Activity

          Luke Lu created issue -
          Hide
          Todd Lipcon added a comment -

          Seems pretty reasonable. Though the proliferation of metrics endpoints is a little strange - a lot of this stuff (but not all) is also available via the hadoop metrics system at /metrics.

          Is there some guiding principle we should follow when deciding whether to add things via JMX vs via Metrics? I always tend towards Metrics since you can use a jmx metrics context to expose them to JMX consumers, whereas the reverse is not true.

          Show
          Todd Lipcon added a comment - Seems pretty reasonable. Though the proliferation of metrics endpoints is a little strange - a lot of this stuff (but not all) is also available via the hadoop metrics system at /metrics. Is there some guiding principle we should follow when deciding whether to add things via JMX vs via Metrics? I always tend towards Metrics since you can use a jmx metrics context to expose them to JMX consumers, whereas the reverse is not true.
          Hide
          Luke Lu added a comment -

          jmx is not just metrics, as the name implies, it's a management interface that can handle metrics, events and other meta info (like software version etc.) and we're still not sure whether we should expose the write/update interface (for changing configs dynamically etc.)

          I'd propose that we deprecate/remove the old /metrics servlet (which needs to be ported to the new metrics system anyway) once /jmx is in place.

          Show
          Luke Lu added a comment - jmx is not just metrics, as the name implies, it's a management interface that can handle metrics, events and other meta info (like software version etc.) and we're still not sure whether we should expose the write/update interface (for changing configs dynamically etc.) I'd propose that we deprecate/remove the old /metrics servlet (which needs to be ported to the new metrics system anyway) once /jmx is in place.
          Hide
          Allen Wittenauer added a comment -

          Did .21 ship with /metrics? If not, there is no need to deprecate it. We can just remove it.

          Show
          Allen Wittenauer added a comment - Did .21 ship with /metrics? If not, there is no need to deprecate it. We can just remove it.
          Hide
          Luke Lu added a comment -

          we're still not sure whether we should expose the write/update interface (for changing configs dynamically etc.)

          ... as part of the servlet interface. We already use JMX directly to change metrics config etc.

          Show
          Luke Lu added a comment - we're still not sure whether we should expose the write/update interface (for changing configs dynamically etc.) ... as part of the servlet interface. We already use JMX directly to change metrics config etc.
          Hide
          Luke Lu added a comment -

          Did .21 ship with /metrics? If not, there is no need to deprecate it. We can just remove it.

          Looks like /metrics is in .21/.22, so we'll have to deprecate it in 0.23+.

          Show
          Luke Lu added a comment - Did .21 ship with /metrics? If not, there is no need to deprecate it. We can just remove it. Looks like /metrics is in .21/.22, so we'll have to deprecate it in 0.23+.
          Hide
          Todd Lipcon added a comment -

          Yea, I'm aware JMX is a superset in functionality compared to Metrics, in that it's not read-only.

          But, metrics is a superset to JMX in that it already has a bunch of different pluggable output libraries - eg Ganglia. Plus we already have metrics integration for lots of things that haven't been JMX-ified yet.

          So I'd prefer to keep exposing data via metrics, and then use a JMX context to also publish to JMX. Then existing users of /metrics, the Ganglia context, etc, won't be broken. Of course adding new read/write fields to JMX makes perfect sense and wouldn't be possible with metrics.

          Show
          Todd Lipcon added a comment - Yea, I'm aware JMX is a superset in functionality compared to Metrics, in that it's not read-only. But, metrics is a superset to JMX in that it already has a bunch of different pluggable output libraries - eg Ganglia. Plus we already have metrics integration for lots of things that haven't been JMX-ified yet. So I'd prefer to keep exposing data via metrics, and then use a JMX context to also publish to JMX. Then existing users of /metrics, the Ganglia context, etc, won't be broken. Of course adding new read/write fields to JMX makes perfect sense and wouldn't be possible with metrics.
          Hide
          Luke Lu added a comment -

          All metrics are published via JMX via HADOOP-6728. Metrics doesn't support status, event and a lot meta information that are not time-series, which is a good thing by design. I'm not proposing deprecating metrics system per se (in fact HADOOP-6728 enhanced it in a major way), but the /metrics servlet.

          Show
          Luke Lu added a comment - All metrics are published via JMX via HADOOP-6728 . Metrics doesn't support status, event and a lot meta information that are not time-series, which is a good thing by design. I'm not proposing deprecating metrics system per se (in fact HADOOP-6728 enhanced it in a major way), but the /metrics servlet.
          Luke Lu made changes -
          Field Original Value New Value
          Assignee Luke Lu [ vicaya ]
          Hide
          Robert Joseph Evans added a comment -

          I have attached an initial patch pulling in JMXProxyServlet.java from Tomcat 7.0.14.
          I pulled over two classes and part of a third.

          org.apache.tomcat.util.ExceptionUtils - only the package name changed.

          org.apache.tomcat.util.modeler.Registry - two methods from this class were moved into JMXProxyServlet getType and convertValue and modified to use mBeanServer from JMXProxyServlet instead of the MBeanServer from Registry

          org.apache.catalina.manager.JMXProxyServlet - pacakge name changed. Registry was removed and replaced with a direct call to get MBeanServer, and two private methods.

          There are no tests and no documentation changes right now (Which is why I marked it alpha). I wanted feedback on it first.

          1) is the package OK org.apache.hadoop.jmx?
          2) is the URL OK /jmx?
          3) where would be an appropriate place to document this?
          4) is this even the right thing to do? I am not sure how standard the format is. I need to dig into it a bit more to really understand it, because all I have done is a simple port.

          Show
          Robert Joseph Evans added a comment - I have attached an initial patch pulling in JMXProxyServlet.java from Tomcat 7.0.14. I pulled over two classes and part of a third. org.apache.tomcat.util.ExceptionUtils - only the package name changed. org.apache.tomcat.util.modeler.Registry - two methods from this class were moved into JMXProxyServlet getType and convertValue and modified to use mBeanServer from JMXProxyServlet instead of the MBeanServer from Registry org.apache.catalina.manager.JMXProxyServlet - pacakge name changed. Registry was removed and replaced with a direct call to get MBeanServer, and two private methods. There are no tests and no documentation changes right now (Which is why I marked it alpha). I wanted feedback on it first. 1) is the package OK org.apache.hadoop.jmx? 2) is the URL OK /jmx? 3) where would be an appropriate place to document this? 4) is this even the right thing to do? I am not sure how standard the format is. I need to dig into it a bit more to really understand it, because all I have done is a simple port.
          Robert Joseph Evans made changes -
          Attachment HADOOP-7411-trunk-alpha.patch [ 12479908 ]
          Hide
          Robert Joseph Evans added a comment -

          Just a quick note, I read through the code for the JMXProxy and it is a plain text format, not JSON or XML.

          It only supports GET but that can be be modified with a ?set&attr=<KEY>&val=<VALUE> to set an attribute; ?get&attr=<KEY> to get a specific attribute; or ?qry=<QUERY> to output all attributes who's names match the query using the mBeanServer.queryNames method. the default is ?qry=: if nothing else if given.

          Show
          Robert Joseph Evans added a comment - Just a quick note, I read through the code for the JMXProxy and it is a plain text format, not JSON or XML. It only supports GET but that can be be modified with a ?set&attr=<KEY>&val=<VALUE> to set an attribute; ?get&attr=<KEY> to get a specific attribute; or ?qry=<QUERY> to output all attributes who's names match the query using the mBeanServer.queryNames method. the default is ?qry= : if nothing else if given.
          Hide
          Luke Lu added a comment -

          is the package OK org.apache.hadoop.jmx? is the URL OK /jmx?

          where would be an appropriate place to document this?

          package-info.java for javadoc would be a start.

          Is this even the right thing to do? I am not sure how standard the format is. I need to dig into it a bit more to really understand it, because all I have done is a simple port.

          We need read-only access and have it return JSON, something like this:

            [ 
              { 
                "bean": "Hadoop:service=...",
                "attrs": {
                  "attr1": "value1",
                  ...
                  "attrn": "valuen"
                }
              },
              ...
            ]
          

          We also need a way to return a pre-configured set of attributes.

          Show
          Luke Lu added a comment - is the package OK org.apache.hadoop.jmx? is the URL OK /jmx? where would be an appropriate place to document this? package-info.java for javadoc would be a start. Is this even the right thing to do? I am not sure how standard the format is. I need to dig into it a bit more to really understand it, because all I have done is a simple port. We need read-only access and have it return JSON, something like this: [ { "bean": "Hadoop:service=...", "attrs": { "attr1": "value1", ... "attrn": "valuen" } }, ... ] We also need a way to return a pre-configured set of attributes.
          Hide
          Luke Lu added a comment -

          is the package OK org.apache.hadoop.jmx? is the URL OK /jmx?

          lgtm.

          Show
          Luke Lu added a comment - is the package OK org.apache.hadoop.jmx? is the URL OK /jmx? lgtm.
          Hide
          Robert Joseph Evans added a comment -

          Ok because we are going to go with JSON, and not use JMXProxyServlet more or less unchanged, I think I will take the core of it, but I will probably rename it so there is no confusion with the Tomcat one.

          Show
          Robert Joseph Evans added a comment - Ok because we are going to go with JSON, and not use JMXProxyServlet more or less unchanged, I think I will take the core of it, but I will probably rename it so there is no confusion with the Tomcat one.
          Robert Joseph Evans made changes -
          Assignee Luke Lu [ vicaya ] Robert Joseph Evans [ revans2 ]
          Hide
          Robert Joseph Evans added a comment -

          This is version 1 of the patch. It includes javadocs, tests, and the servlet.

          The servlet is placed under /jmx and returns data in a JSON format. Example output is attached too.

          Show
          Robert Joseph Evans added a comment - This is version 1 of the patch. It includes javadocs, tests, and the servlet. The servlet is placed under /jmx and returns data in a JSON format. Example output is attached too.
          Robert Joseph Evans made changes -
          Attachment HADOOP-7411-0.20.20X-V1.patch [ 12480458 ]
          Attachment HADOOP-7411-trunk-V1.patch [ 12480459 ]
          Attachment jmx.json [ 12480460 ]
          Robert Joseph Evans made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Luke Lu added a comment -

          +1, lgtm, let's see hudson/jenkins' +1. Thanks for the json output (looks like it's 203?) as well, I spotted a bug (HADOOP-7330) the metrics system JMX code that I thought I fixed but didn't go into 203.

          Show
          Luke Lu added a comment - +1, lgtm, let's see hudson/jenkins' +1. Thanks for the json output (looks like it's 203?) as well, I spotted a bug ( HADOOP-7330 ) the metrics system JMX code that I thought I fixed but didn't go into 203.
          Hide
          Robert Joseph Evans added a comment -

          The JSON is from a task tracker on 20 security trunk, with the name of the box replaced with foo

          http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20-security
          Revision: 1127672

          It is targeted for 0.20.205

          Show
          Robert Joseph Evans added a comment - The JSON is from a task tracker on 20 security trunk, with the name of the box replaced with foo http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20-security Revision: 1127672 It is targeted for 0.20.205
          Hide
          Luke Lu added a comment -

          The JSON is from a task tracker on 20 security trunk, with the name of the box replaced with foo

          Thanks for the precise answer. I meant whether the output is from 0.20.20x.

          Show
          Luke Lu added a comment - The JSON is from a task tracker on 20 security trunk, with the name of the box replaced with foo Thanks for the precise answer. I meant whether the output is from 0.20.20x.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12480460/jmx.json
          against trunk revision 1127811.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/526//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12480460/jmx.json against trunk revision 1127811. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/526//console This message is automatically generated.
          Hide
          Robert Joseph Evans added a comment -

          Ahh, Really Jenkins tried to use the JSON as a patch? Oh well the following are the manual results for trunk
          [exec] +1 overall.
          [exec]
          [exec] +1 @author. The patch does not contain any @author tags.
          [exec]
          [exec] +1 tests included. The patch appears to include 4 new or modified tests.
          [exec]
          [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
          [exec]
          [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
          [exec]
          [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
          [exec]
          [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings.
          [exec]
          [exec] +1 system test framework. The patch passed system test framework compile.
          [exec]

          Show
          Robert Joseph Evans added a comment - Ahh, Really Jenkins tried to use the JSON as a patch? Oh well the following are the manual results for trunk [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 4 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 system test framework. The patch passed system test framework compile. [exec]
          Hide
          Robert Joseph Evans added a comment -

          Hey Luke, what will it take to get HADOOP-7330 into 0.20.20X? I looked at the patch and it looks straight forward? It would be nice to have the JSON output consistent between releases.

          Show
          Robert Joseph Evans added a comment - Hey Luke, what will it take to get HADOOP-7330 into 0.20.20X? I looked at the patch and it looks straight forward? It would be nice to have the JSON output consistent between releases.
          Hide
          Luke Lu added a comment -

          what will it take to get HADOOP-7330 into 0.20.20X?

          +1 on the jira and we'll nag one of the committers to push the changes to these branches

          Show
          Luke Lu added a comment - what will it take to get HADOOP-7330 into 0.20.20X? +1 on the jira and we'll nag one of the committers to push the changes to these branches
          Hide
          Owen O'Malley added a comment -

          I don't think the exception handling is appropriate. Why do you need to catch them at all?

          You certainly don't want to silently swallow Errors or mix their handling with IOExceptions.

          Show
          Owen O'Malley added a comment - I don't think the exception handling is appropriate. Why do you need to catch them at all? You certainly don't want to silently swallow Errors or mix their handling with IOExceptions.
          Hide
          Robert Joseph Evans added a comment -

          It is there because that is what tomcat is doing inside their JMXProxy Servlet which I modified, and didn't take the time to understand all of the possible exceptions that could be thrown in each of the cases to clean it up.

          Part of the reason for ignoring the exceptions is because JMX can execute arbitrary code inside the JMXBeans, which could throw all kinds of exceptions. Ideally we would like the code to be robust so that if some beans are causing issues that we can still output data for the rest of the beans. But I will look at what it takes to clean up the exception handling.

          Show
          Robert Joseph Evans added a comment - It is there because that is what tomcat is doing inside their JMXProxy Servlet which I modified, and didn't take the time to understand all of the possible exceptions that could be thrown in each of the cases to clean it up. Part of the reason for ignoring the exceptions is because JMX can execute arbitrary code inside the JMXBeans, which could throw all kinds of exceptions. Ideally we would like the code to be robust so that if some beans are causing issues that we can still output data for the rest of the beans. But I will look at what it takes to clean up the exception handling.
          Hide
          Luke Lu added a comment -

          LOG.error is always called before handleThrowable. It looked reasonable (it's not swallowing all throwables and log the errors) to me at the time, as we should never let jetty handle the exceptions/errors as stack traces would be display as part of the result, which is a security no no.

          OTOH, I think the logging should be handled by handleThrowable calls for better readability and DRYness.

          Looking closely though, I found follow correctness issues:

          1. content type should be set to "application/json; charset=utf8"
          2. need to response.setStatus(response.SC_INTERNAL_SERVER_ERROR) for all error cases
          Show
          Luke Lu added a comment - LOG.error is always called before handleThrowable. It looked reasonable (it's not swallowing all throwables and log the errors) to me at the time, as we should never let jetty handle the exceptions/errors as stack traces would be display as part of the result, which is a security no no. OTOH, I think the logging should be handled by handleThrowable calls for better readability and DRYness. Looking closely though, I found follow correctness issues: content type should be set to "application/json; charset=utf8" need to response.setStatus(response.SC_INTERNAL_SERVER_ERROR) for all error cases
          Robert Joseph Evans made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Hide
          Robert Joseph Evans added a comment -

          I believe that I have addressed all of the comments. The code no longer catches a Throwable, but enumerates the errors that can occur at each location, and handles them appropriately. IOExceptions are passed up to doGet which will return an INTERNAL SERVER ERROR. If the query is malformed then a BAD REQUEST is returned. In no case is the json being used to return the error.

          Show
          Robert Joseph Evans added a comment - I believe that I have addressed all of the comments. The code no longer catches a Throwable, but enumerates the errors that can occur at each location, and handles them appropriately. IOExceptions are passed up to doGet which will return an INTERNAL SERVER ERROR. If the query is malformed then a BAD REQUEST is returned. In no case is the json being used to return the error.
          Robert Joseph Evans made changes -
          Attachment HADOOP-7411-0.20.20X-V2.patch [ 12481099 ]
          Attachment HADOOP-7411-trunk-V2.patch [ 12481100 ]
          Hide
          Robert Joseph Evans added a comment -

          The following are the results of test-patch on the security branch.

          [exec] -1 overall.
          [exec]
          [exec] +1 @author. The patch does not contain any @author tags.
          [exec]
          [exec] +1 tests included. The patch appears to include 4 new or modified tests.
          [exec]
          [exec] -1 javadoc. The javadoc tool appears to have generated 1 warning messages.
          [exec]
          [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
          [exec]
          [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings.
          [exec]
          [exec] -1 Eclipse classpath. The patch causes the Eclipse classpath to differ from the contents of the lib directories.

          The eclipse path is bogus, and so is the findbugs. I don't know why they keep complaining but they do.

          Show
          Robert Joseph Evans added a comment - The following are the results of test-patch on the security branch. [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 4 new or modified tests. [exec] [exec] -1 javadoc. The javadoc tool appears to have generated 1 warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] -1 Eclipse classpath. The patch causes the Eclipse classpath to differ from the contents of the lib directories. The eclipse path is bogus, and so is the findbugs. I don't know why they keep complaining but they do.
          Robert Joseph Evans made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12481100/HADOOP-7411-trunk-V2.patch
          against trunk revision 1129989.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 4 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/555//testReport/
          Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/555//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/555//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12481100/HADOOP-7411-trunk-V2.patch against trunk revision 1129989. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 4 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/555//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/555//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-HADOOP-Build/555//console This message is automatically generated.
          Hide
          Luke Lu added a comment -

          +1. It's good enough for commit, even though IMO, it's better to catch and log Throwable and setStatus 500 for security reasons (it's usually recommended to disallow stack traces in responses in production mode, although it's not a big deal in this particular case). BTW, if you submit the trunk patch last and make patch available, hudson/jenkins will pickup the right patch.

          Show
          Luke Lu added a comment - +1. It's good enough for commit, even though IMO, it's better to catch and log Throwable and setStatus 500 for security reasons (it's usually recommended to disallow stack traces in responses in production mode, although it's not a big deal in this particular case). BTW, if you submit the trunk patch last and make patch available, hudson/jenkins will pickup the right patch.
          Hide
          Luke Lu added a comment -

          Another improvement would be to setStatus 404 if the queryNames result is empty.

          Show
          Luke Lu added a comment - Another improvement would be to setStatus 404 if the queryNames result is empty.
          Hide
          Chris Douglas added a comment -

          +1

          I committed this to trunk and the 20-security branch. Thanks, Robert!

          Show
          Chris Douglas added a comment - +1 I committed this to trunk and the 20-security branch. Thanks, Robert!
          Chris Douglas made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Fix Version/s 0.20.205.0 [ 12316390 ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #639 (See https://builds.apache.org/hudson/job/Hadoop-Common-trunk-Commit/639/)
          HADOOP-7144. Expose JMX metrics via JSON servlet.
          Contributed by Robert Joseph Evans

          cdouglas : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1131289
          Files :

          • /hadoop/common/trunk/src/java/org/apache/hadoop/http/HttpServer.java
          • /hadoop/common/trunk/CHANGES.txt
          • /hadoop/common/trunk/src/java/org/apache/hadoop/jmx
          • /hadoop/common/trunk/src/test/core/org/apache/hadoop/jmx/TestJMXJsonServlet.java
          • /hadoop/common/trunk/src/java/org/apache/hadoop/jmx/JMXJsonServlet.java
          • /hadoop/common/trunk/src/java/org/apache/hadoop/jmx/package-info.java
          • /hadoop/common/trunk/src/test/core/org/apache/hadoop/jmx
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #639 (See https://builds.apache.org/hudson/job/Hadoop-Common-trunk-Commit/639/ ) HADOOP-7144 . Expose JMX metrics via JSON servlet. Contributed by Robert Joseph Evans cdouglas : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1131289 Files : /hadoop/common/trunk/src/java/org/apache/hadoop/http/HttpServer.java /hadoop/common/trunk/CHANGES.txt /hadoop/common/trunk/src/java/org/apache/hadoop/jmx /hadoop/common/trunk/src/test/core/org/apache/hadoop/jmx/TestJMXJsonServlet.java /hadoop/common/trunk/src/java/org/apache/hadoop/jmx/JMXJsonServlet.java /hadoop/common/trunk/src/java/org/apache/hadoop/jmx/package-info.java /hadoop/common/trunk/src/test/core/org/apache/hadoop/jmx
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk #709 (See https://builds.apache.org/hudson/job/Hadoop-Common-trunk/709/)
          HADOOP-7144. Expose JMX metrics via JSON servlet.
          Contributed by Robert Joseph Evans

          cdouglas : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1131289
          Files :

          • /hadoop/common/trunk/src/java/org/apache/hadoop/http/HttpServer.java
          • /hadoop/common/trunk/CHANGES.txt
          • /hadoop/common/trunk/src/java/org/apache/hadoop/jmx
          • /hadoop/common/trunk/src/test/core/org/apache/hadoop/jmx/TestJMXJsonServlet.java
          • /hadoop/common/trunk/src/java/org/apache/hadoop/jmx/JMXJsonServlet.java
          • /hadoop/common/trunk/src/java/org/apache/hadoop/jmx/package-info.java
          • /hadoop/common/trunk/src/test/core/org/apache/hadoop/jmx
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk #709 (See https://builds.apache.org/hudson/job/Hadoop-Common-trunk/709/ ) HADOOP-7144 . Expose JMX metrics via JSON servlet. Contributed by Robert Joseph Evans cdouglas : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1131289 Files : /hadoop/common/trunk/src/java/org/apache/hadoop/http/HttpServer.java /hadoop/common/trunk/CHANGES.txt /hadoop/common/trunk/src/java/org/apache/hadoop/jmx /hadoop/common/trunk/src/test/core/org/apache/hadoop/jmx/TestJMXJsonServlet.java /hadoop/common/trunk/src/java/org/apache/hadoop/jmx/JMXJsonServlet.java /hadoop/common/trunk/src/java/org/apache/hadoop/jmx/package-info.java /hadoop/common/trunk/src/test/core/org/apache/hadoop/jmx
          Hide
          Suresh Srinivas added a comment -

          Is the JMX provided in this jira exposes the document equivalent to the MBean or an MBeam method?

          If it is entire MBean, then this can cause issues with Namenode where live node list, dead node list, decom node list etc are returned in one huge json document. Should the access be per mbean property?

          Show
          Suresh Srinivas added a comment - Is the JMX provided in this jira exposes the document equivalent to the MBean or an MBeam method? If it is entire MBean, then this can cause issues with Namenode where live node list, dead node list, decom node list etc are returned in one huge json document. Should the access be per mbean property?
          Hide
          Luke Lu added a comment -

          Should the access be per mbean property?

          The initial customers for this jira are ops querying metrics mbeans, so the per mbean size was not an issue. I agree that for things like liveNodes from the NameNodeInfo mbean, returning all the attributes of the mbean is too wasteful for mere access of lighter-weight attributes in the same mbean.

          Let's open a separate jira to add the get=attribute-or-pattern mechanism.

          Show
          Luke Lu added a comment - Should the access be per mbean property? The initial customers for this jira are ops querying metrics mbeans, so the per mbean size was not an issue. I agree that for things like liveNodes from the NameNodeInfo mbean, returning all the attributes of the mbean is too wasteful for mere access of lighter-weight attributes in the same mbean. Let's open a separate jira to add the get=attribute-or-pattern mechanism.
          Hide
          Tanping Wang added a comment -

          Yes, we want to be able to query each individual property of a mbean using JMXJsonServerlet. Something like

          http://hostname/jmx?qry=mbeanName:property

          Show
          Tanping Wang added a comment - Yes, we want to be able to query each individual property of a mbean using JMXJsonServerlet. Something like http://hostname/jmx?qry=mbeanName:property
          Hide
          Tanping Wang added a comment -

          Suppose we can query each property of a mbean, can we return the JSON string exactly like what the mbean property function returns? This way it is consistent between querying through JMXJsonServlet and JMX itself. No additional parsing will be needed between both situations, it will just be a plug-and-play when switching back and forth.

          For example,

          	NameNode#getLiveNodes() 
          

          returns a Json string like,

          "LiveNodes" : "{\"l.yahoo.com\":{\"usedSpace\":49152,\"lastContact\":2,\"adminState\":\"In Service\"},\"4.yahoo.com\":{\"usedSpace\":49152,\"lastContact\":0,\"adminState\":\"Decommissioned\"}}"
          

          It would be helpful if the exact Json string can be returned when querying via JMXJsonServlet. If possible, can we eliminate the bean name and modelerType from the returned string like how JMXJsonServerlet is currently returning, i.e.

          {
            "beans" : [ {
              "name" : "Hadoop:service=NameNode,name=NameNodeInfo",
              "modelerType" : "org.apache.hadoop.hdfs.server.namenode.FSNamesystem",
              "Threads" : 34,
          ....
          
          Show
          Tanping Wang added a comment - Suppose we can query each property of a mbean, can we return the JSON string exactly like what the mbean property function returns? This way it is consistent between querying through JMXJsonServlet and JMX itself. No additional parsing will be needed between both situations, it will just be a plug-and-play when switching back and forth. For example, NameNode#getLiveNodes() returns a Json string like, "LiveNodes" : "{\"l.yahoo.com\":{\"usedSpace\":49152,\"lastContact\":2,\"adminState\":\"In Service\"},\"4.yahoo.com\":{\"usedSpace\":49152,\"lastContact\":0,\"adminState\":\"Decommissioned\"}}" It would be helpful if the exact Json string can be returned when querying via JMXJsonServlet. If possible, can we eliminate the bean name and modelerType from the returned string like how JMXJsonServerlet is currently returning, i.e. { "beans" : [ { "name" : "Hadoop:service=NameNode,name=NameNodeInfo", "modelerType" : "org.apache.hadoop.hdfs.server.namenode.FSNamesystem", "Threads" : 34, ....
          Tanping Wang made changes -
          Link This issue is part of HADOOP-7392 [ HADOOP-7392 ]
          Hide
          Tanping Wang added a comment -

          created HADOOP-7392 to implement querying per mbean property.

          Show
          Tanping Wang added a comment - created HADOOP-7392 to implement querying per mbean property.
          Owen O'Malley made changes -
          Fix Version/s 0.20.204.0 [ 12316317 ]
          Owen O'Malley made changes -
          Fix Version/s 0.20.205.0 [ 12316390 ]
          Hide
          Owen O'Malley added a comment -

          Hadoop 0.20.204.0 was released today.

          Show
          Owen O'Malley added a comment - Hadoop 0.20.204.0 was released today.
          Owen O'Malley made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Patch Available Patch Available Open Open
          6d 17h 54m 1 Robert Joseph Evans 01/Jun/11 16:20
          Open Open Patch Available Patch Available
          99d 18h 20m 2 Robert Joseph Evans 01/Jun/11 16:24
          Patch Available Patch Available Resolved Resolved
          2d 8h 17m 1 Chris Douglas 04/Jun/11 00:41
          Resolved Resolved Closed Closed
          90d 22h 33m 1 Owen O'Malley 02/Sep/11 23:15

            People

            • Assignee:
              Robert Joseph Evans
              Reporter:
              Luke Lu
            • Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development