Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1408

Herriot NN and DN clients should vend statistics


    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: test
    • Labels:
    • Hadoop Flags:
    • Tags:


      The HDFS web user interface serves useful information through dfshealth.jsp and dfsnodelist.jsp.

      The Herriot interface to the namenode and datanode (as implemented in NNClient and DNClient, respectively) would benefit from the addition of some way to channel this information. In the case of DNClient this can be an injected method that returns a DatanodeDescriptor relevant to the underlying datanode.

      There seems to be no analagous NamenodeDescriptor. It may be useful to add this as a facade to a visitor that aggregates values across the filesystem datanodes. These values are (from dfshealth JSP):

      Configured Capacity
      DFS Used
      Non DFS Used
      DFS Remaining
      DFS Used%
      DFS Remaining%
      Live Nodes
      Dead Nodes
      Decommissioning Nodes
      Number of Under-Replicated Blocks

      Attributes reflecting the web user interface header may also be useful such as When-Started, Version, When-Compiled, and Upgrade-Status.

      A NamenodeDescriptor would essentially "push down" the code in dfshealth web UI behind a more general abstraction. If it is objectionable to make this class available in HDFS, perhaps this could be packaged in a Herriot specific way.

      1. HDFS-1408.patch
        4 kB
        Konstantin Boudnik
      2. HDFS-1408.patch
        5 kB
        Konstantin Boudnik
      3. HDFS-1408.patch
        3 kB
        Konstantin Boudnik
      4. add_JMXlisteners.patch
        6 kB
        Konstantin Boudnik
      5. jmx.patch
        15 kB
        Konstantin Boudnik
      6. jmx.patch
        19 kB
        Konstantin Boudnik
      7. jmx.patch
        18 kB
        Konstantin Boudnik
      8. jmx.patch
        7 kB
        Konstantin Boudnik
      9. HDFS-1408.patch
        8 kB
        Konstantin Boudnik
      10. HADOOP-6927.y20s.patch
        8 kB
        Konstantin Boudnik

        Issue Links



            • Assignee:
              Konstantin Boudnik
              Al Thompson
            • Votes:
              0 Vote for this issue
              1 Start watching this issue


              • Created: