[HIVE-26155] Create a new connection pool for compaction

• Type: Improvement
• Status: In Progress
• Priority: Major
• Resolution: Unresolved
• Component/s: Standalone Metastore
• Labels: compaction pull-request-available
• Assignee: Antal Sinkovits
• Reporter: Antal Sinkovits
• Created: 19/Apr/22 18:06
• Updated: 28/Apr/22 10:33

Description

Currently the TxnHandler uses 2 connection pools to communicate with the HMS: the default one and one for mutexing. If compaction is configured incorrectly (e.g. too many Initiators are running on the same db) then compaction can use up all the connections in the default connection pool and all user queries can get stuck.

We should have a separate connection pool (configurable size) just for compaction-related activities.

Issue Links

• links to: GitHub Pull Request #3223

Sub-Tasks

• Create a new connection pool for compaction (CompactionTxnHandler): Resolved
• Create a new connection pool for compaction (DataNucleus): Resolved