[HDDS-3286] BasicOzoneFileSystem support batchDelete

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.0.0
• Component/s: Ozone Filesystem
• Labels: Triaged pull-request-available
• Target Version/s: 1.0.0
• Assignee: mingchao zhao
• Reporter: mingchao zhao
• Created: 27/Mar/20 03:35
• Updated: 27/Jul/20 23:33
• Resolved: 23/Jun/20 21:22

Description

    Currently delete file is to get all the keys in the directory, and then delete one by one. And the same with rename. This makes for poor performance.

    By tested the deletion path with 100,000 files, which took 3718.70 sec. And rename it took 7327.936.

    HDDS-2939 can able to optimize this part, but at present the hdds-2939 is slow and still a long way to go. So we optimized the batch operation based on the current interface. We were able to get better performance with this PR before the hdds-2939 came in.

Issue Links

• contains: HDDS-3620 BasicOzoneFileSystem support batchRename
• is depended upon by: HDDS-4040 [OFS] BasicRootedOzoneFileSystem to support batchDelete
• links to: GitHub Pull Request #814