Affects Version/s: 0.18.2
Fix Version/s: 0.21.0
Hadoop Flags:Incompatible change, Reviewed
Release Note:Changed df dfsadmin -report to list live and dead nodes, and attempt to resolve the hostname of datanode ip addresses.
As part of operations responsibility to bring back dead nodes, it will be good to have a quick way to obtain a list of dead data nodes.
The current way is to scrape the namenode web UI page and parse that information, but this creates load on the namenode.
In search of a less costly way, I noticed dfsadmin -report only reports data nodes with State: "In Service" and "Decommission in progress" get listed.
Asking for a cheap way to obtain a list of dead nodes.
In addition, can the following requests be reviewed for additional enhancement and changes to dfsadmin -report.
- Consistent formatting output in "Remaining raw bytes:" for the data nodes should have a space between the exact value and the parenthesized value.
Total raw bytes: 3842232975360 (3.49 TB)
Remaining raw bytes: 146090593065(136.06 GB)
Used raw bytes: 3240864964620 (2.95 TB)
- Include the running version of Hadoop.
- What is the meaning of "Total effective bytes"?
- Display the hostname instead of the IP address for the data node (toggle option?)