[HDDS-7541] FSO recursive delete directory with hierarchy takes much time for cleanup

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 1.4.0
• Component/s: OM
• Labels: pull-request-available
• Target Version/s: 1.4.0, 1.3.1
• Assignee: Sumit Agrawal
• Reporter: Sumit Agrawal
• Created: 25/Nov/22 04:49
• Updated: 14/Mar/23 21:35
• Resolved: 05/Dec/22 06:38

Description

When level of hierarchy is high, it takes (nrOfLevel*1Minute) time which is quiet high for cleanup.

This can be done quickly within number of files range configuration within iteration.

 

cleanup log of directory (each iteration with 1 minute):

2022-11-15 20:05:13,854 INFO service.DirectoryDeletingService (DirectoryDeletingService.java:call(217)) - Number of dirs deleted: 1, Number of sub-files moved: 0 to DeletedTable, Number of sub-dirs moved 1 to DeletedDirectoryTable, iteration elapsed: 14ms, totalRunCount: 9
2022-11-15 20:06:13,856 INFO service.DirectoryDeletingService (DirectoryDeletingService.java:call(217)) - Number of dirs deleted: 1, Number of sub-files moved: 1 to DeletedTable, Number of sub-dirs moved 1 to DeletedDirectoryTable, iteration elapsed: 12ms, totalRunCount: 10
2022-11-15 20:07:13,959 INFO service.DirectoryDeletingService (DirectoryDeletingService.java:call(217)) - Number of dirs deleted: 1, Number of sub-files moved: 0 to DeletedTable, Number of sub-dirs moved 1 to DeletedDirectoryTable, iteration elapsed: 17ms, totalRunCount: 11
2022-11-15 20:08:13,972 INFO service.DirectoryDeletingService (DirectoryDeletingService.java:call(217)) - Number of dirs deleted: 1, Number of sub-files moved: 0 to DeletedTable, Number of sub-dirs moved 0 to DeletedDirectoryTable, iteration elapsed: 26ms, totalRunCount: 12 

Issue Links

• duplicates: HDDS-5048 [FSO] Perf optimization - Recursively visit sub-path components and do cleanup
• relates to: HDDS-7328 Improve Deletion of FSO Paths
• links to: GitHub Pull Request #4008