[HIVE-25777] ACID: Pick the compactor transaction over insert dir

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.2, 4.0.0
• Component/s: Transactions
• Labels: Compaction pull-request-available
• Assignee: Denys Kuzmenko
• Reporter: Gopal Vijayaraghavan
• Created: 06/Dec/21 17:50
• Updated: 30/Aug/22 09:34
• Resolved: 30/Aug/22 09:34

Description

If there are two competing versions of a particular write-id, one from the compactor and another from the original insert, always pick the compactor one once it is committed.

If the directory structure looks like

base_000011/
base_000011_v192/

Then always pick the v192 transaction if txnid=192 is committed.

This is required to ensure that the raw base_ dir can be deleted safely on non-atomic directory deletions (like s3), without a race condition between getSplits and the actual file-reader.

Issue Links

• links to: GitHub Pull Request #2968