[BIGTOP-3963] merge hive memory leak PRs

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.2.0
• Fix Version/s: 3.3.0
• Component/s: hive
• Assignee: Robin Zuo
• Reporter: Robin Zuo
• Created: 25/Jul/23 06:52
• Updated: 08/May/24 00:20
• Resolved: 08/May/24 00:20

Description

Currently, Hive3 has some issues, especially following memory leak related issues, which have been blocked the usage of hive in production environment due to frequently OOM. For this reason, I'd like to merge them in patch format to bigtop project.

 

They are:
HIVE-16455: ADD JAR command leaks JAR Files
https://issues.apache.org/jira/browse/HIVE-16455
HIVE-24236/HIVE-25483: Connection leak in TxnHandler
https://issues.apache.org/jira/browse/HIVE-24236
https://issues.apache.org/jira/browse/HIVE-25483
HIVE-24552: Possible HMS connections leak or accumulation in loadDynamicPartitions
https://issues.apache.org/jira/browse/HIVE-24552

HIVE-24858: UDFClassLoader leak in Configuration.CACHE_CLASSES

https://issues.apache.org/jira/browse/HIVE-24858
HIVE-26404: HMS memory leak when compaction cleaner fails to remove obsolete files
https://issues.apache.org/jira/browse/HIVE-26404
HIVE-26530: HS2 OOM-OperationManager.queryIdOperation does not properly clean up multiple queryIds (This PR has been fixed in hive-3.1.3)
https://issues.apache.org/jira/browse/HIVE-26530