[HDDS-4504] Datanode deletion config should be based on number of blocks

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.2.0
• Component/s: Ozone Datanode
• Labels: pull-request-available
• Target Version/s: 1.2.0
• Assignee: Aryan Gupta
• Reporter: Lokesh Jain
• Created: 24/Nov/20 10:10
• Updated: 27/Apr/21 08:01
• Resolved: 05/Apr/21 08:09

Description

Currently datanode's deletion config is based on number of containers on which deletion should be performed(default value 10). The deletion is only performed on those containers.

It should rather be based on number of blocks to delete per interval rather than number of containers to process. If a datanode has 100000 containers and every container has 10 blocks to delete then deletion of all these blocks can take a lot of time with the current configuration. It would take 100000/10 = 10000 intervals for deletion to complete.

Issue Links

• breaks: HDDS-5113 Intermittent failure in TestBlockDeletingService#testContainerThrottle
• breaks: HDDS-5147 Intermittent test failure in TestContainerDeletionChoosingPolicy#testRandomChoosingPolicy
• links to: GitHub Pull Request #1885