[ARROW-12153] [Rust] [Parquet] Return file metadata after writing Parquet file

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0
• Component/s: Rust
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/27974
• Assignee: Neville Dipale
• Reporter: Neville Dipale
• Created: 30/Mar/21 16:34
• Updated: 11/Jan/23 08:24
• Resolved: 31/Mar/21 16:47

Description

Parquet writers like delta-rs rely on the Parquet metadata to write file-level statistics for file pruning purposes.

We currently do not expose these stats, requiring the writer to read the file that has just been written, to get the stats. This is more problematic for in-memory sinks, as there is currently no way of getting the metadata from the sink before it's persisted.

Explore if we can expose these stats to the writer, to make the above easier.

Issue Links

• links to: GitHub Pull Request #9850