[IMPALA-11047] Preconditions.checkNotNull(statsTuple_) fail in HdfsScanNode.java if PARQUET_READ_STATISTICS=0

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 4.0.0
• Fix Version/s: Impala 4.1.0
• Target Version: Impala 4.0.0
• Epic Color: ghx-label-3
• Assignee: Qifan Chen
• Reporter: Riza Suminto
• Created: 03/Dec/21 20:39
• Updated: 19/Aug/22 23:23
• Resolved: 24/Feb/22 17:12

Description

There is a conflict happening in HdfsScanNode.java vs RuntimeFilterGenerator.java when initializing overlap predicate.

In HdfsScanNode.java, computeStatsTupleAndConjuncts that init statsTuple_ will not be called because PARQUET_READ_STATISTICS=0.
https://github.com/apache/impala/blob/9d61bc4/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java#L409

 

On the other hand, in RuntimeFilterGenerator.java, disable_overlap_filter is set to false without considering what is the value of PARQUET_READ_STATISTICS.
https://github.com/apache/impala/blob/9d61bc4/fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java#L915

Issue Links

• is related to: IMPALA-11071 NPE when PARQUET_READ_STATISTICS is set to false
• is related to: IMPALA-11294 Remove unnecessary workarounds from dictionary runtime filter tests