[ARROW-5342] [Format] Formalize extension type metadata in IPC protocol

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.14.0
• Component/s: Format
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/21801
• Assignee: Wes McKinney
• Reporter: Wes McKinney
• Created: 15/May/19 22:03
• Updated: 11/Jan/23 07:39
• Resolved: 14/Jun/19 13:19

Description

On the heels of ARROW-585 and ARROW-5255, it has been proposed to add reserved keys to custom_metadata on the Field Flatbuffer type to pass along the name of a custom/user-defined data type and any serialized data required to reconstruct that data type.

The objective of this is to enable library users to define new data types that are embedded semantically in pre-defined Arrow data types like Binary or FixedSizeBinary. For example, UUID could be defined as an extension type atop FixedSizeBinary(16) (though UUID may be important enough to formalize as a logical type at some point...)

I will propose language to add to Metadata.rst and start a discussion on the mailing list to possibly call a vote

Issue Links

• relates to: ARROW-5255 [Java] Implement user-defined data types API
• relates to: ARROW-585 [C++] Define public API for user-defined data types
• links to: GitHub Pull Request #4332