[ARROW-4314] [Rust] Strongly-typed reading of Parquet data

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Invalid
• Component/s: Rust
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/20886
• Assignee: Unassigned
• Reporter: Alec Mocatta
• Created: 21/Jan/19 21:59
• Updated: 11/Jan/23 07:33
• Resolved: 26/Apr/21 11:21

Description

See the proposal I made on csun's repository here for more details.

This aims to let the user opt in to strong typing and substantial performance improvements (2x-7x, see here) by optionally specifying the type of the records that they are iterating over.

It is currently a work in progress. All pre-existing tests succeed, bar those in src/record/api.rs which are commented out as they require reworking. Where relevant, pre-existing tests and benchmarks have been duplicated to make new strongly-typed tests and benchmarks, which all also succeed. I've tried to maintain pre-existing APIs where possible. Some changes have been made to better align with prior art in the Rust ecosystem.

Any feedback while I continue working on it very welcome! Looking forward to hopefully seeing this merged when it's ready.

Issue Links

• links to: GitHub Pull Request #3461