[HDFS-13831] Make block increment deletion number configurable

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.0
• Fix Version/s: 2.10.0, 3.2.0, 3.0.4, 3.1.2
• Target Version/s: 2.10.0, 3.2.0
• Hadoop Flags: Reviewed
• Assignee: Ryan Wu
• Reporter: Yiqun Lin
• Created: 17/Aug/18 03:20
• Updated: 22/Apr/21 09:52
• Resolved: 28/Aug/18 23:37

Description

When NN deletes a large directory, it will hold the write lock long time. For improving this, we remove the blocks in a batch way. So that other waiters have a chance to get the lock. But right now, the batch number is a hard-coded value.

  static int BLOCK_DELETION_INCREMENT = 1000;

We can make this value configurable, so that we can control the frequency of other waiters to get the lock chance.

Issue Links

• is related to: HDFS-15994 Deletion should sleep some time, when there are too many pending deletion blocks.