[PARQUET-2413] Support custom file footer metadata via ParquetWriter

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 1.14.0
• Assignee: Claire McGinty
• Reporter: Claire McGinty
• Created: 11/Dec/23 18:38
• Updated: 23/Jun/24 03:33
• Resolved: 28/Jan/24 07:27

Description

Currently Parquet supports extra metadata keys passed via WriteContext. However, in practice these are limited to implementation-specific values (i.e. AvroWriteSupport adds a key containing the serialized Avro schema), and cannot be overridden or extended by the end user.

 

Maybe we could extend the ParquetWriter builder to support this? For example:

 

```

AvroParquetWriter.Builder<T> builder = AvroParquetWriter.<T>builder(outputFile)

    .withConf(...)

    .withPageSize(...)

+  .withExtraMetadata(Map.of(..))

    .build()

```

Issue Links

• links to: GitHub Pull Request #1241