[HIVE-14350] Aborted txns cause false positive "Not enough history available..." msgs

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.3.0, 2.1.1
• Fix Version/s: 1.3.0, 2.1.1, 2.2.0
• Component/s: Transactions
• Assignee: Eugene Koifman Assign to me
• Reporter: Eugene Koifman
• Created: 27/Jul/16 02:47
• Updated: 08/Dec/16 14:15
• Resolved: 01/Aug/16 19:26

Description

this is a followup to HIVE-13369. Only open txns should prevent use of a base file. But ValidTxnList does not make a distinction between open and aborted txns. The presence of aborted txns causes false positives which can happen too often since the flow is
1. Worker generates a new base file,
2. then asynchronously Cleaner removes now-compacted aborted txns. (strictly speaking it's Initiator that does the actual clean up)

So we may have base_5 and base_10 and txnid 7 aborted. Then current impl will disallow use of base_10 though there is no need for that. Worse, if txnid_4 is aborted and hasn't been purged yet, base_5 will be rejected as well and then an error will be raised since there is no suitable base file left.

ErrorMsg.ACID_NOT_ENOUGH_HISTORY is msg produced

Issue Links

• is related to: HIVE-14865 Fix comments after HIVE-14350
• relates to: HIVE-14414 Fix TestHiveMetaStoreTxns UTs
• requires: HIVE-13369 AcidUtils.getAcidState() is not paying attention toValidTxnList when choosing the "best" base file