[HDFS-1408] Herriot NN and DN clients should vend statistics

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.22.0
• Fix Version/s: 0.22.0
• Component/s: test
• Hadoop Flags: Reviewed
• Tags: herriot
• Assignee: Konstantin I Boudnik
• Reporter: Al Thompson
• Created: 26/Aug/10 16:48
• Updated: 12/Dec/11 06:20
• Resolved: 11/Nov/10 19:44

Description

The HDFS web user interface serves useful information through dfshealth.jsp and dfsnodelist.jsp.

The Herriot interface to the namenode and datanode (as implemented in NNClient and DNClient, respectively) would benefit from the addition of some way to channel this information. In the case of DNClient this can be an injected method that returns a DatanodeDescriptor relevant to the underlying datanode.

There seems to be no analagous NamenodeDescriptor. It may be useful to add this as a facade to a visitor that aggregates values across the filesystem datanodes. These values are (from dfshealth JSP):

Configured Capacity
DFS Used
Non DFS Used
DFS Remaining
DFS Used%
DFS Remaining%
Live Nodes
Dead Nodes
Decommissioning Nodes
Number of Under-Replicated Blocks

Attributes reflecting the web user interface header may also be useful such as When-Started, Version, When-Compiled, and Upgrade-Status.

A NamenodeDescriptor would essentially "push down" the code in dfshealth web UI behind a more general abstraction. If it is objectionable to make this class available in HDFS, perhaps this could be packaged in a Herriot specific way.

Issue Links

• is blocked by: HADOOP-6977 Herriot daemon clients should vend statistics
• is cloned by: HADOOP-6977 Herriot daemon clients should vend statistics
• is cloned by: MAPREDUCE-2093 Herriot JT and TT clients should vend statistics