[ARROW-11410] [Rust][Parquet] Implement returning dictionary arrays from parquet reader

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Invalid
• Component/s: Rust
• External issue URL: https://github.com/apache/arrow/issues/27299
• Assignee: Unassigned
• Reporter: Yordan Pavlov
• Created: 27/Jan/21 22:26
• Updated: 11/Jan/23 08:19
• Resolved: 26/Apr/21 12:46

Description

Currently the Rust parquet reader returns a regular array (e.g. string array) even when the column is dictionary encoded in the parquet file.

If the parquet reader had the ability to return dictionary arrays for dictionary encoded columns this would bring many benefits such as:

• faster reading of dictionary encoded columns from parquet (as no conversion/expansion into a regular array would be necessary)
• more efficient memory use as the dictionary array would use less memory when loaded in memory
• faster filtering operations as SIMD can be used to filter over the numeric keys of a dictionary string array instead of comparing string values in a string array

nevime , alamb  let me know what you think