Absolutely excellent to have a web UI here; thanks for making it!
A few comments on the JMX approach:
(1) The namenode already has a port open for the Namenode RPC protocol and another one open for HTTP. Have you considered adding a getWhatever() API to the namenode RPC protocol or adding a Jetty servlet endpoint for the same? Either approach would require less configuration. We already have two perfectly good protocols; why bother with a third? (
(2) You provide a way to set the agent port, but, ultimately, there are a lot of properties involved in setting up JMX (http://download.oracle.com/javase/6/docs/technotes/guides/management/agent.html has ones for security, SSL, passwords and credentials, etc.) As you know, once you connect via JMX, you can do pretty much anything, including injecting code into the namenode. It seems that setting '-Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false' (as you do in one of the patches in the subissues) by default is unwise from a security perspective.
(3) The method you use to get the config from hdfs-site to the shell script ends up creating an extra JVM. That's not terrible: it's not that Hadoop startup time is particularly speedy to begin with, but we already have options in hadoop-env.sh (like heapsize), so the extra mechanism feels like it may be gratuitous. I looked around to see if there was a way to start (or open) the JMX agent port programatically, but I did not find one.