[ARROW-13191] [Go] Support external schema in ipc readers

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: Go
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/28881
• Assignee: Unassigned
• Reporter: Seth Hollyman
• Created: 25/Jun/21 22:30
• Updated: 11/Jan/23 08:31

Description

(Apologies if I'm imprecise here, I'm still coming up to speed on the arrow details.)

 

The IPC message format describes how data and metadata messages are encapsulated, but it is not a requirement that each message must include the schema.

 

In Go, github.com/apache/arrow/go/arrow/ipc contains NewReader() for setting up reading of IPC messages, and accepts the option WithSchema to pass the schema into said reader.  However, the implementation merely uses that information to compare that the schema it reads from the IPC stream matches the passed in reader.  This request is to allow WithSchema to behave as expected, and use the option-provided Schema for performing reads.

 

The one gotcha here appears to be the dictionary type map, which is currently retained independently of the schema but is part of the internal readSchema() setup. Completeness may warrant another option for communicating those externally as well?  Or perhaps option-passed Schema should be documented to not support dictionary types?

Issue Links

• links to: GitHub Pull Request #10603