[ARROW-10636] [Rust] Remove specialisation from Rust parquet

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0
• Component/s: Rust
• Labels: parquet pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/26592
• Assignee: Greg Bowyer
• Reporter: Greg Bowyer
• Created: 18/Nov/20 03:36
• Updated: 11/Jan/23 08:14
• Resolved: 15/Dec/20 20:58

Description

This is a very initial attempt at removing the specialization features from the Rust Parquet implementation.

The specialisation is too complex to be covered by min_specialization and requires a bit of reworking in the crate.

Right now the code dispatches in sub-traits and methods on the Parquet type, and uses a combination of trait abuse, macros and transmutes to eliminate the feature.

I have broken this up into several commits ranging from the simplest removals (which could probably be taken fairly easily) to the most ugly and complex.

I am not stoked on the transmute abuse, and I think another take (or follow up) should be taken to remove as many as possible in the code.

The general trait for DataType::T has been made a private sealed trait to make it impossible to implement external to the Parquet crate, this is intentional as I dont think many of the public interfaces are sensible for end users to be able to implement.

TODO:

• Purge the added {{std::mem::transmute}}s if possible
• Refine and rationalise the unimplemented! implementations
• Performance test?

Issue Links

• is related to: ARROW-10931 [Rust] [Parquet] Improve performance of the parquet compressors
• relates to: ARROW-6717 [Rust] Support stable Rust
• links to: GitHub Pull Request #8698