[ARROW-10135] [Rust] [Parquet] Refactor file module to help adding sources

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.1
• Fix Version/s: 3.0.0
• Component/s: Rust
• Labels: parquet pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/26146
• Assignee: Rémi Dettai
• Reporter: Rémi Dettai
• Created: 29/Sep/20 13:51
• Updated: 11/Jan/23 08:11
• Resolved: 25/Oct/20 20:25

Description

Currently, the Parquet reader is very strongly tied to file system reads. This makes it hard to add other sources. For instance, to implement S3, we would need a reader that loads entire columns at once rather than buffered reads of a few Ko.

To improve modularity, we could try to move as much logic as possible to the generic traits (FileReader, RowGroupReader...) and reduce the code in the implementing structs (SerializedFileReader, SerializedRowGroupReader...) to the part that is specific to file/buffered reads.

Issue Links

• links to: GitHub Pull Request #8300