[HIVE-14988] Support INSERT OVERWRITE into a partition on transactional tables

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.0
• Fix Version/s: 3.0.0
• Component/s: Transactions
• Target Version/s: 3.0.0
• Assignee: Wei Zheng
• Reporter: Eugene Koifman
• Created: 17/Oct/16 18:25
• Updated: 23/May/18 00:00
• Resolved: 17/Jul/17 06:12

Description

Insert overwrite operation on transactional table will currently raise an error.
This can/should be supported

This is implemented by using the concept of base_N/ directory in Acid.
INSERT OVERWRITE starts a new transaction N and creates a new base_N directory where it writes the data.
This has the effect that on read all files with transaction IDs < N are ignored.
The advantage is that IOW doesn't need to be a mutexed operation. All readers that stared before this IOW can proceed at the
snapshot (Acid tables support at Snapshot Isolation) they locked in.

If mutexed semantics are desired we could add an option where IOW will request an X lock but the base_N/ based implementation can support both.

Issue Links

• is related to: HIVE-18021 Insert overwrite on acid table with Union All optimizations
• is related to: HIVE-17774 compaction may start with 0 splits and fail
• is related to: HIVE-18589 java.io.IOException: Not enough history available
• is related to: HIVE-17457 IOW Acid Insert Overwrite when the transaction fails
• is related to: HIVE-18131 Truncate table for Acid tables
• relates to: HIVE-18154 IOW Acid Load Data/Insert with Overwrite in multi statement transactions
• links to: Review Board