Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-1408

Herriot NN and DN clients should vend statistics

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Tags:
      herriot

      Description

      The HDFS web user interface serves useful information through dfshealth.jsp and dfsnodelist.jsp.

      The Herriot interface to the namenode and datanode (as implemented in NNClient and DNClient, respectively) would benefit from the addition of some way to channel this information. In the case of DNClient this can be an injected method that returns a DatanodeDescriptor relevant to the underlying datanode.

      There seems to be no analagous NamenodeDescriptor. It may be useful to add this as a facade to a visitor that aggregates values across the filesystem datanodes. These values are (from dfshealth JSP):

      Configured Capacity
      DFS Used
      Non DFS Used
      DFS Remaining
      DFS Used%
      DFS Remaining%
      Live Nodes
      Dead Nodes
      Decommissioning Nodes
      Number of Under-Replicated Blocks

      Attributes reflecting the web user interface header may also be useful such as When-Started, Version, When-Compiled, and Upgrade-Status.

      A NamenodeDescriptor would essentially "push down" the code in dfshealth web UI behind a more general abstraction. If it is objectionable to make this class available in HDFS, perhaps this could be packaged in a Herriot specific way.

        Attachments

        1. add_JMXlisteners.patch
          6 kB
          Konstantin Boudnik
        2. HADOOP-6927.y20s.patch
          8 kB
          Konstantin Boudnik
        3. HDFS-1408.patch
          4 kB
          Konstantin Boudnik
        4. HDFS-1408.patch
          5 kB
          Konstantin Boudnik
        5. HDFS-1408.patch
          3 kB
          Konstantin Boudnik
        6. HDFS-1408.patch
          8 kB
          Konstantin Boudnik
        7. jmx.patch
          15 kB
          Konstantin Boudnik
        8. jmx.patch
          19 kB
          Konstantin Boudnik
        9. jmx.patch
          18 kB
          Konstantin Boudnik
        10. jmx.patch
          7 kB
          Konstantin Boudnik

          Issue Links

            Activity

              People

              • Assignee:
                cos Konstantin Boudnik
                Reporter:
                almacro Al Thompson
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: