[HDFS-11789] Maintain Short-Circuit Read Statistics

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0-alpha4
• Component/s: hdfs-client
• Target Version/s: 2.9.0
• Hadoop Flags: Reviewed
• Assignee: Hanisha Koneru
• Reporter: Hanisha Koneru
• Created: 09/May/17 23:12
• Updated: 22/Jun/17 20:59
• Resolved: 22/Jun/17 20:37

Description

If a disk or controller hardware is faulty then short-circuit read requests can stall indefinitely while reading from the file descriptor. Currently there is no way to detect when short-circuit read requests are slow or blocked.

This Jira proposes that each BlockReaderLocal maintain read statistics while it is active by measuring the time taken for a pre-determined fraction of read requests. These per-reader stats can be aggregated into global stats when the reader is closed. The aggregate statistics can be exposed via JMX.

Issue Links

• requires: HADOOP-14503 Make RollingAverages a mutable metric