[HIVE-20890] ACID: Allow whole table ReadLocks to skip all partition locks

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-2
• Component/s: Transactions
• Labels: pull-request-available
• Assignee: Denys Kuzmenko
• Reporter: Gopal Vijayaraghavan
• Created: 08/Nov/18 07:04
• Updated: 16/Nov/22 13:50
• Resolved: 29/Jun/22 12:12

Description

HIVE-19369 proposes adding a EXCL_WRITE lock which does not wait for any SHARED_READ locks for read operations - in the presence of that lock, the insert overwrite no longer takes an exclusive lock.

The only exclusive operation will be a schema change or drop table, which should take an exclusive lock on the entire table directly.

explain locks select * from tpcds_bin_partitioned_orc_1000.store_sales where ss_sold_date_sk=2452626 

+----------------------------------------------------+
|                      Explain                       |
+----------------------------------------------------+
| LOCK INFORMATION:                                  |
| tpcds_bin_partitioned_orc_1000.store_sales -> SHARED_READ |
| tpcds_bin_partitioned_orc_1000.store_sales.ss_sold_date_sk=2452626 -> SHARED_READ |
+----------------------------------------------------+

So the per-partition SHARED_READ locks are no longer necessary, if the lock builder already includes the table-wide SHARED_READ locks.

The removal of entire partitions is the only part which needs to be taken care of within this semantics as row-removal instead of directory removal (i.e "drop partition" -> "truncate partition" and have the truncation trigger a whole directory cleaner, so that the partition disappears when there are 0 rows left).

Issue Links

• depends upon: HIVE-19369 Locks: Add new lock implementations for always zero-wait readers
• is duplicated by: HIVE-23714 Add new configuration for lock escalation
• is related to: HIVE-21354 Lock The Entire Table If Majority Of Partitions Are Locked
• is related to: HIVE-26165 Remove READ locks for ACID tables
• links to: GitHub Pull Request #1166