[HIVE-19598] Add Acid V1 to V2 upgrade module

• Type: Sub-task
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.1.0
• Component/s: Transactions
• Target Version/s: 3.1.0
• Assignee: Eugene Koifman
• Reporter: Eugene Koifman
• Created: 17/May/18 21:04
• Updated: 31/Jul/18 01:33
• Resolved: 31/May/18 20:37

Description

The on-disk layout for full acid (transactional) tables has changed 3.0.

Any transactional table that has any update/delete events in any deltas that have not been Major compacted, must go through a Major compaction before upgrading to 3.0.  No more update/delete/merge should be run after/during major compaction.

Not doing so will result in data corruption/loss.

 

Need to create a utility tool to help with this process.  HIVE-19233 started this but it needs more work.

Issue Links

• is required by: HIVE-19751 create submodule of hive-upgrade-acid for preUpgrade and postUpgrade
• relates to: HIVE-19599 Release Notes : Highlighting backwards incompatible changes
• relates to: HIVE-19233 Add utility for acid 1.0 to 2.0 migration
• supercedes: HIVE-19553 Enable TestTxnExIm#testUpgrade
• links to: RB