[ARROW-15552] [Docs][Format] Unclear wording about base64 encoding requirement of metadata values

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 8.0.0
• Component/s: Documentation, Format
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/20100
• Assignee: Antoine Pitrou
• Reporter: Joris Van den Bossche
• Created: 03/Feb/22 16:16
• Updated: 11/Jan/23 11:37
• Resolved: 08/Feb/22 10:36

Description

The C Data Interface docs indicate that the values in key-value metadata should be base64 encoded, which is mentioned in the section about which key-value metadata to use for extension types (https://arrow.apache.org/docs/format/CDataInterface.html#extension-arrays):

The base64 encoding of metadata values ensures that any possible serialization is representable.

This might not be fully correct, though (or at least not required, which is implied with the current wording). While a binary blob (like a serialized schema) can be base64 encoded, as we do when putting the Arrow schema in the Parquet metadata, this is not required?

cc apitrou

Issue Links

• links to: GitHub Pull Request #12361