[HDFS-16582] Expose aggregate latency of slow node as perceived by the reporting node

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0, 3.3.5
• Fix Version/s: 3.4.0, 3.3.5
• Component/s: metrics
• Labels: pull-request-available
• Target Version/s: 3.4.0, 3.3.5
• Assignee: Viraj Jasani
• Reporter: Viraj Jasani
• Created: 18/May/22 04:20
• Updated: 27/Jan/24 01:25
• Resolved: 21/May/22 02:14

Description

When any datanode is reported to be slower by another node, we expose the slow node as well as the reporting nodes list for the slow node. However, we don't provide latency numbers of the slownode as reported by the reporting node. Having the latency exposed in the metrics would be really helpful for operators to keep a track of how far behind a given slow node is performing compared to the rest of the nodes in the cluster.

The operator should be able to gather aggregated latencies of all slow nodes with their reporting nodes in Namenode metrics.

Issue Links

• is related to: HDFS-11194 Maintain aggregated peer performance metrics on NameNode
• is related to: HDFS-16521 DFS API to retrieve slow datanodes
• relates to: HDFS-16595 Slow peer metrics - add median, mad and upper latency limits
• links to: GitHub Pull Request #4323