[HIVE-25115] Compaction queue entries may accumulate in "ready for cleaning" state

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Labels: pull-request-available
• Assignee: Denys Kuzmenko
• Reporter: Karen Coppage
• Created: 14/May/21 08:09
• Updated: 19/Jul/22 15:07
• Resolved: 02/Aug/21 07:43

Description

If the Cleaner does not delete any files, the compaction queue entry is thrown back to the queue and remains in "ready for cleaning" state.

Problem: If 2 compactions run on the same table and enter "ready for cleaning" state at the same time, only one "cleaning" will remove obsolete files, the other entry will remain in the queue in "ready for cleaning" state.

Issue Links

• causes: HIVE-25781 Restore multi-threaded support in Cleaner after HIVE-25115
• relates to: HIVE-24314 compactor.Cleaner should not set state "mark cleaned" if it didn't remove any files
• links to: GitHub Pull Request #2277
• links to: GitHub Pull Request #2764