[HIVE-27408] Parquet file opened for reading stats is never closed

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-beta-1
• Labels: pull-request-available
• Assignee: Sourabh Badhya
• Reporter: Sourabh Badhya
• Created: 04/Jun/23 04:46
• Updated: 15/Aug/23 06:33
• Resolved: 06/Jun/23 07:32

Description

ParquetRecordWriterWrapper while closing the writer tries to collect the stats by creating a reader (opening the file). But it never closes the reader (never closes the file). This can leave the file open hence consuming memory and associated file handle resources.

https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/ParquetRecordWriterWrapper.java#L143-L155

Issue Links

• links to: GitHub Pull Request #4387