[HDFS-17538] Add tranfer priority queue for decommissioning datanode

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Assignee: Unassigned
• Reporter: Yuanbo Liu
• Created: 29/May/24 08:28
• Updated: 03/Jun/24 06:51
• Resolved: 03/Jun/24 06:51

Description

When decommissioning datanode, blocks will be checked one by one disk, then blocks will be sent to trigger tranfer works in DN. This will make one disk of decommissioning dn very busy and cpus stuck in io-wait with high loads, and sometime even lead to OOM as below:

Proposal to add priority queue for transfering blocks when decommisioning datanode.

Issue Links

• is duplicated by: HDFS-14854 Create improved decommission monitor implementation