[IMPALA-4123] Columnar decoding in Parquet scanner

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 2.8.0
• Fix Version/s: Impala 3.2.0
• Component/s: Backend
• Labels: parquet performance
• Target Version: Product Backlog
• Assignee: Tim Armstrong
• Reporter: Tim Armstrong
• Created: 13/Sep/16 19:56
• Updated: 07/Feb/23 11:12
• Resolved: 17/Nov/18 19:45

Description

Apache Parquet have some nice performance improvements to bit-packed decoding in their Parquet scanner (which is derived from Impala's): https://github.com/apache/parquet-cpp/pull/140

We should do something similar - i.e. switch to more of a batch-oriented approach to decoding rather than value-at-a-time

Issue Links

• is a parent of: IMPALA-4177 Add batch dictionary/RLE decoding in Parquet
• is related to: IMPALA-4864 Speed up binary predicates against dictionary encoded Parquet data by converting the predicates to their codewords

Sub-Tasks

• Add batched bit-unpacking utility functions: Resolved