[ARROW-3769] [C++] Support reading non-dictionary encoded binary Parquet columns directly as DictionaryArray

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.13.0
• Component/s: C++
• Labels: parquet pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/20103
• Assignee: Hatem Helal
• Reporter: Wes McKinney
• Created: 25/Sep/18 11:25
• Updated: 11/Jan/23 07:29
• Resolved: 18/Mar/19 00:13

Description

If the goal is to hash this data anyway into a categorical-type array, then it would be better to offer the option to "push down" the hashing into the Parquet read hot path rather than first fully materializing a dense vector of ByteArray values, which could use a lot of memory after decompression

Issue Links

• is related to: ARROW-3325 [Python] Support reading Parquet binary/string columns directly as DictionaryArray
• relates to: PARQUET-1508 [C++] Enable reading from ByteArray and FixedLenByteArray decoders directly into arrow::BinaryBuilder or arrow::BinaryDictionaryBuilder
• links to: GitHub Pull Request #3721