[ARROW-1964] [Python] Expose Builder classes

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.10.0
• Component/s: Python
• Labels: beginner pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/17951
• Assignee: Donal Simmie
• Reporter: Uwe Korn
• Created: 04/Jan/18 08:01
• Updated: 11/Jan/23 07:18
• Resolved: 10/May/18 15:29

Description

Having the builder classes available from Python would be very helpful. Currently a construction of an Arrow array always need to have a Python list or numpy array as intermediate. As the builder in combination with jemalloc are very efficient in building up non-chunked memory, it would be nice to directly use them in certain cases.

The most useful builders are the StringBuilder and DictionaryBuilder as they provide functionality to create columns that are not easily constructed using NumPy methods in Python.

The basic approach would be to wrap the C++ classes in https://github.com/apache/arrow/blob/master/python/pyarrow/includes/libarrow.pxd so that they can be used from Cython. Afterwards, we should start a new file python/pyarrow/builder.pxi where we have classes take typical Python objects like str and pass them on to the C++ classes. At the end, these classes should also return (Python accessible) pyarrow.Array instances.

Issue Links

• links to: GitHub Pull Request #1930