[HIVE-19490] Locking on Insert into for non native and managed tables.

• Type: Improvement
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 3.1.0, 4.0.0-alpha-1
• Component/s: Druid integration
• Labels: druid locking
• Target Version/s: 3.1.0
• Assignee: Slim Bouguerra
• Reporter: Slim Bouguerra
• Created: 10/May/18 17:15
• Updated: 27/Feb/24 22:23
• Resolved: 28/May/18 16:39

Description

Current state of the art: 

Managed non native table like Druid Tables, will need to get a Lock on Insert into or insert Over write. The nature of this lock is set to Exclusive by default for any non native table.

This implies that Inserts into Druid table will Lock any read query as well during the execution of the insert into. IMO this lock (on insert into) is  not needed since the insert statement is appending data and the state of loading it is managed partially by Hive Storage handler hook and part of it by Druid. 

What i am proposing is to relax the lock level to shared for all non native tables on insert into operations and keep it as Exclusive Write for insert Overwrite for now.

 

Any feedback is welcome.

cc ekoifman / ashutoshc / jdere / hagleitn

Also am not sure what is the best way to unit test this currently am using debugger to check if locks are what i except, please let me know if there is a better way to do this.