[HIVE-25781] Restore multi-threaded support in Cleaner after HIVE-25115

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Denys Kuzmenko
• Created: 07/Dec/21 09:40
• Updated: 17/Nov/22 08:47
• Resolved: 22/Dec/21 16:35

Description

In the case of accumulated entries for the same table/partition, they should be processed one by one in ASC order of write_id's.
However, to support multi-threaded processing in the Cleaner, we need to make sure that the entries from the same group are not processed concurrently by multiple threads.
One of the solutions that don't involve synchronization (check if already in flight, re-try when possible) would be to move the entries from the same group to the next Cleaner cycle.

Issue Links

• is caused by: HIVE-25115 Compaction queue entries may accumulate in "ready for cleaning" state
• links to: GitHub Pull Request #2825