[HIVE-16360] Improve "No delta files or original files found to compact in" message

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.2.0
• Fix Version/s: 3.0.0
• Component/s: Transactions
• Target Version/s: 3.0.0
• Assignee: Eugene Koifman
• Reporter: Eugene Koifman
• Created: 03/Apr/17 18:20
• Updated: 01/Oct/19 22:07
• Resolved: 26/Sep/17 16:32

Description

CompactorMR.run() produces

LOG.error("No delta files or original files found to compact in " + sd.getLocation() + " for compactionId=" + ci.id);
      

in some cases even where there are plenty of delta directories in the partition.
The reason is that compactor won't compact anything above the level of an open transaction. So if there is a very long running txn, it may filter out all deltas in a partition. This looks very confusing when looking at the logs.

Issue Links

• is related to: HIVE-12636 Ensure that all queries (with DbTxnManager) run in a transaction