[HIVE-19369] Locks: Add new lock implementations for always zero-wait readers

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Component/s: Transactions
• Assignee: Denys Kuzmenko
• Reporter: Gopal Vijayaraghavan
• Created: 01/May/18 01:06
• Updated: 28/Apr/20 10:47
• Resolved: 24/Apr/20 07:27

Description

Hive Locking with Micro-managed and full-ACID tables needs a better locking implementation which allows for no-wait readers always.

EXCL_DROP
EXCL_WRITE
SHARED_WRITE
SHARED_READ

Short write-up

EXCL_DROP is a "drop partition" or "drop table" and waits for all others to exit
EXCL_WRITE excludes all writes and will wait for all existing SHARED_WRITE to exit.
SHARED_WRITE allows all SHARED_WRITES to go through, but will wait for an EXCL_WRITE & EXCL_DROP (waiting so that you can do drop + insert in different threads).

SHARED_READ does not wait for any lock - it fails fast for a pending EXCL_DROP, because even if there is an EXCL_WRITE or SHARED_WRITE pending, there's no semantic reason to wait for them to succeed before going ahead with a SHARED_WRITE.

a select * => SHARED_READ
an insert into => SHARED_WRITE
an insert overwrite or MERGE => EXCL_WRITE
a drop table => EXCL_DROP

TODO:

The fate of the compactor needs to be added to this before it is a complete description.

Issue Links

• contains: HIVE-23270 Optimize isValidTxnListState to reduce the numbers of HMS calls
• fixes: HIVE-23259 Fix Apache Rat validation
• is depended upon by: HIVE-20890 ACID: Allow whole table ReadLocks to skip all partition locks
• is related to: HIVE-18570 ACID IOW implemented using base may delete too much data

Sub-Tasks

• Locks: Implement zero-wait readers: Closed