[ARROW-18423] [Python] Expose reading a schema from an IPC message

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 11.0.0
• Component/s: Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/33586
• Assignee: Andre Kohn
• Reporter: Andre Kohn
• Created: 02/Dec/22 23:03
• Updated: 11/Jan/23 12:00
• Resolved: 14/Dec/22 14:38

Description

Pyarrow currently does not implement the reading of an Arrow schema from an IPC message.

https://github.com/apache/arrow/blob/80b389efe902af376a85a8b3740e0dbdc5f80900/python/pyarrow/ipc.pxi#L1094

 

We'd like to consume an Arrow IPC stream like the following:

 

schema_msg = pyarrow.ipc.read_message(result_iter.next().data)
schema = pyarrow.ipc.read_schema(schema_msg)
for batch_data in result_iter:
    batch_msg = pyarrow.ipc.read_message(batch_data.data)
    batch = pyarrow.ipc.read_record_batch(batch_msg, schema)

 

The associated (tiny) PR on GitHub implements this reading by binding the existing C++ function.

Issue Links

• links to: GitHub Pull Request #14831