[HDDS-4344] Block Deletion Performance Improvements

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Target Version/s: 1.5.0
• Assignee: Lokesh Jain
• Reporter: Lokesh Jain
• Created: 14/Oct/20 09:58
• Updated: 16/Dec/23 23:43

Description

In cluster deployments it was observed that block deletion can be slow. For example if a user writes a million keys in Ozone, the time it takes for those million keys to be deleted from datanodes can be high. The jira would cover various improvements which can be made for better deletion speeds.

Sub-Tasks

• Allow multiple transactions per container to be sent for deletion by SCM: Resolved
• SCM deletion service should delete configured number of blocks every interval: Resolved
• Configuration for deletion service intervals should be different for OM, SCM and datanodes: Resolved
• Datanode should store the delete transaction as is in rocksDB: Resolved
• Datanode deletion service can avoid storing deleted blocks: Resolved
• SCM should create transactions using all blocks received from OM: Resolved
• Avoid counting skipped transactions belonging to open containers: Resolved
• With HA OM can send deletion blocks to SCM multiple times: Resolved
• SCM can avoid using RETRIABLE_DATANODE_COMMAND for datanode deletion commands: Resolved
• Datanode deletion config should be based on number of blocks: Resolved
• SCM can avoid creating RetriableDatanodeEventWatcher for deletion command ACK: Resolved
• Increase default value for key and block deletion limit: Resolved
• SCM should avoid sending delete transactions for under-replicated containers: Resolved
• Datanode should have deletion service thread per volume: Open
• Datanode delete operations should not impact rocksDB cache: Open
• SCM should reduce RPC payLoad to OM for deletion ACK: Open