[DRILL-6164] Heap memory leak during parquet scan and OOM

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.13.0
• Labels: ready-to-commit
• Assignee: Vlad Rozov
• Reporter: Vlad Rozov
• Reviewer: Parth Chandra
• Created: 16/Feb/18 02:29
• Updated: 19/Feb/18 09:33
• Resolved: 19/Feb/18 09:33

Description

During a scan of a large set of parquet files, Drill iterates over the set initializing parquet readers. Such initialization may require a significant memory usage (both heap and direct). When scan moves to the next parquet file in the set, it does not remove reference to the reader from the set it iterates over and does not remove references created during initialization.

Issue Links

• links to: GitHub Pull Request #1122