[HDFS-15994] Deletion should sleep some time, when there are too many pending deletion blocks.

• Type: Improvement
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Assignee: Qi Zhu
• Reporter: Qi Zhu
• Created: 22/Apr/21 09:33
• Updated: 07/May/21 14:08

Description

HDFS-13831 realize that we can control the frequency of other waiters to get the lock chance.

But actually in our big cluster with heavy deletion:

The problem still happened,  and the pending deletion blocks will be more than ten million somtimes, and the size become more than 1 million in regular in huge clusters.

So i think we should sleep for some time when pending too many deletion blocks. 

Issue Links

• relates to: HDFS-13831 Make block increment deletion number configurable