[HIVE-18193] Migrate existing ACID tables to use write id per table rather than global transaction id

• Type: Sub-task
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.1.0, 3.0.0
• Component/s: HiveServer2, Transactions
• Labels: ACID Upgrade
• Target Version/s: 3.0.0
• Assignee: Sankar Hariappan
• Reporter: Anishek Agarwal
• Created: 01/Dec/17 06:28
• Updated: 22/May/18 23:13
• Resolved: 11/May/18 19:01

Description

dependent upon HIVE-18192

For existing ACID Tables we need to update the table level write id metatables/sequences so any new operations on these tables works seamlessly without any conflicting data in existing base/delta files.

1. Need to create metadata tables such as NEXT_WRITE_ID and TXN_TO_WRITE_ID.

2. Add entries for each ACID/MM tables into NEXT_WRITE_ID where NWI_NEXT is set to current value of NEXT_TXN_ID.NTXN_NEXT.

3. All current open/abort transactions to have an entry in TXN_TO_WRITE_ID such that T2W_TXNID=T2W_WRITEID=Open/AbortedTxnId.

4. Added new column TC_WRITEID in TXN_COMPONENTS and CTC_WRITEID in COMPLETED_TXN_COMPONENTS to store the write id which should be set as respective values of TC_TXNID and CTC_TXNID from the same row.

Issue Links

• depends upon: HIVE-18192 Introduce WriteID per table rather than using global transaction ID
• is duplicated by: HIVE-18194 Migrate existing ACID tables to use write id per table rather than global transaction id
• links to: GitHub Pull Request #339