[HDFS-15754] Create packet metrics for DataNode

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 3.4.0
• Component/s: datanode
• Labels: pull-request-available
• Hadoop Flags: Reviewed
• Assignee: Fengnan Li
• Reporter: Fengnan Li
• Created: 29/Dec/20 22:56
• Updated: 08/Jan/21 07:47
• Resolved: 08/Jan/21 07:47

Description

In BlockReceiver, right now when there is slowness in writeToMirror, writeToDisk and writeToOsCache, it is dumped in the debug log. In practice we have found these are quite useful signal to detect issues in DataNode, so it will be great these metrics can be exposed by JMX.
Also we introduced totalPacket received to use a percentage as a signal to detect the potentially underperforming datanode since datanodes across one HDFS cluster may received different numbers of packets totally.

Issue Links

• links to: GitHub Pull Request #2578

Sub-Tasks

• Add metrics for slow DataNode: Resolved