[IMPALA-2735] Push down conjunct evaluation into Parquet column readers

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: Impala 2.3.0
• Component/s: Backend
• Labels: parquet performance
• Target Version: Product Backlog
• Assignee: Michael Ho
• Reporter: Tim Armstrong
• Created: 03/Dec/15 18:26
• Updated: 13/Mar/20 16:24

Description

Currently EvalConjuncts() operates on an entire row, which requires that values be materialised into the row format before evaluating conjuncts. In some cases we partially materialise each tuple before evaluating conjuncts, but this is not cache-efficient if we want to materialise column-by-column.

In some cases, e.g. the Parquet scanner, it would be helpful if we could evaluate conjuncts that only reference a single slot on a simple value outside of the row format so that we can materialise a dense array of that columns values then filter out rows that don't pass conjuncts.

Issue Links

• blocks: IMPALA-2736 Column-wise value materialisation in Parquet scanner
• is related to: IMPALA-4864 Speed up binary predicates against dictionary encoded Parquet data by converting the predicates to their codewords
• is related to: IMPALA-3841 Avoid materializing nested collections if top-level predicates already disqualify the row.
• is related to: IMPALA-8077 Avoid converting timestamps in dropped rows during Parquet scanning