[ARROW-362] Python: Calling to_pandas on a table read from Parquet leaks memory

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.1.0
• Fix Version/s: 0.2.0
• Component/s: Python
• External issue URL: https://github.com/apache/arrow/issues/15907
• Assignee: Wes McKinney
• Reporter: Uwe Korn
• Created: 03/Nov/16 16:27
• Updated: 11/Jan/23 07:08
• Resolved: 07/Nov/16 18:53

Description

Steps to reproduce:

• Read a parquet file with pyarrow.parquet.read_table and convert the table to a DataFrame with to_pandas
• Repeat this several times and see an ever increasing memory usage

This seems to happen only in this combination. Calling gc.collect doesn't help.