[HIVE-27444] Limit items processed by Initiator/Cleaner

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-beta-1
• Component/s: Hive
• Labels: pull-request-available
• Assignee: Zsolt Miskolczi
• Reporter: Zsolt Miskolczi
• Created: 15/Jun/23 13:20
• Updated: 15/Aug/23 06:33
• Resolved: 07/Aug/23 08:50

Description

The list of potential compaction items processed by Initiator is currently unlimited, and this may result in very long Initator loops. To avoid this, the potential compaction items returned by

org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler#findPotentialCompactions(int, long, long)

 

should be limited.

The same limit needs to be added to cleaner:

 

org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler#findReadyToClean
org.apache.hadoop.hive.metastore.txn.CompactionTxnHandler#findReadyToCleanAborts

 

Issue Links

• links to: GitHub Pull Request #4421