[ARROW-457] Python: Better control over memory pool

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.2.0
• Component/s: Python
• External issue URL: https://github.com/apache/arrow/issues/16101
• Assignee: Uwe Korn
• Reporter: Uwe Korn
• Created: 06/Jan/17 14:52
• Updated: 11/Jan/23 07:08
• Resolved: 04/Feb/17 21:23

Description

Currently we have a separate PyArrowMemoryPool implemented in src/pyarrrow/common.cc/h. Instead we should use the default memory pool from Arrow-C++ as often as possible.

Furthermore the user should be able to configure which MemoryPool is actually used in the cases where one can select a custom MemoryPool. For ease of use, there should also be a way to switch the default MemoryPool in Python to a user-selected one, e.g. the jemalloc::MemoryPool.

Issue Links

• is depended upon by: ARROW-458 [Python] Expose jemalloc MemoryPool