[ARROW-6667] [Python] Avoid Reference Cycles in pyarrow.parquet

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 0.15.0
• Component/s: Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/23017
• Assignee: Aaron Opfer
• Reporter: Aaron Opfer
• Created: 23/Sep/19 16:00
• Updated: 11/Jan/23 07:48
• Resolved: 25/Sep/19 10:15

Description

Reference cycles appear in two places inside pyarrow.parquet which causes these objects to have much longer lifetimes than necessary:

 

_build_nested_path has a reference cycle because the closured function refers to the parent cell which also refers to the closured function again (objgraph shown in attachment)

open_dataset_file is partialed with self inside the ParquetFile class (objgraph shown in attachment).

Issue Links

• links to: GitHub Pull Request #5476