[HIVE-18570] ACID IOW implemented using base may delete too much data

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Component/s: Transactions
• Target Version/s: 3.0.0
• Release Note: Insert Overwrite commands even on transactional tables will acquire Exclusive locks to ensure correctness. This will be improved upon to allow greater concurrency.
• Assignee: Eugene Koifman
• Reporter: Sergey Shelukhin
• Created: 29/Jan/18 19:42
• Updated: 04/May/18 20:24
• Resolved: 04/May/18 20:24

Description

Suppose we have a table with delta_0 insert data.
Txn 1 starts an insert into delta_1.
Txn 2 starts an IOW into base_2.
Txn 2 commits.
Txn 1 commits after txn 2 but its results would be invisible.

Txn 2 deletes rows committed by txn 1 that according to standard ACID semantics it could have never observed and affected; this sequence of events is only possible under read-uncommitted isolation level (so, 2 deletes rows written by 1 before 1 commits them).
This is if we look at IOW as transactional delete+insert. Otherwise we are just saying IOW performs "semi"-transactional delete.

If 1 ran an update on rows instead of an insert, and 2 still ran an IOW/delete, row lock conflict (or equivalent) should cause one of them to fail.

Issue Links

• is required by: HIVE-19124 implement a basic major compactor for MM tables
• relates to: HIVE-19369 Locks: Add new lock implementations for always zero-wait readers
• requires: HIVE-18948 Acquire locks before generating the valid transaction list