[HDFS-14988] HDFS should avoid read/write data from slow disks.

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.1.1, 3.2.1
• Component/s: block placement, datanode
• Target Version/s: 3.2.1
• Assignee: Unassigned
• Reporter: yimeng
• Created: 14/Nov/19 01:54
• Updated: 14/Nov/19 01:54

Description

Slow disk causes real-time service(such as HBase ) become slowdown.

The slow disk detection is added to the HDFS-11461, but only the slow disk is recorded in the Metric. I hope to further handle the detected slow disk. In my view, the slow disk can be added to the read data policy. If the block is on the slow disk of the DataNode, the block of other DataNodes is selected. For write data, slow disks can be added to the data write policy. We can remove the slow disk from all disks and then select a disk to write data based on dfs.datanode.fsdataset.volume.choosing.policy.