[ARROW-5931] [C++] Extend extension types facility to provide for serialization and deserialization in IPC roundtrips

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: C++
• External issue URL: https://github.com/apache/arrow/issues/22340
• Assignee: Unassigned
• Reporter: Wes McKinney
• Created: 12/Jul/19 17:55
• Updated: 11/Jan/23 07:43

Description

A use case here is when an array needs to reference some external data. For example, suppose that we wanted to implement an array that references a sequence of Python objects as PyObject*. Obviously, a PyObject* must be managed by the Python interpreter.

For a vector of some T* to be sent through the IPC machinery, it must be embedded in some Arrow type on the wire. For example, the memory resident version of PyObject* might be 8-bytes per value (1 pointer per value) while being serialized to the binary IPC protocol, such PyObject* values must be serialized into an Arrow Binary type.

Issue Links

• relates to: ARROW-5929 [Java] Define API for ExtensionVector whose data must be serialized prior to being sent via IPC