[IMPALA-11071] NPE when PARQUET_READ_STATISTICS is set to false

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: Impala 4.0.0
• Component/s: Frontend
• Epic Color: ghx-label-8
• Assignee: Unassigned
• Reporter: Tamas Mate
• Created: 10/Jan/22 11:56
• Updated: 10/Jan/22 13:08
• Resolved: 10/Jan/22 13:08

Description

A NullPointerException happens when the PARQUET_READ_STATISTICS is set to false, the overlap predicate initialisation is triggered, but the stats are not available.

Repro query:

SET PARQUET_READ_STATISTICS=false;
SELECT * FROM functional_parquet.alltypes a
  JOIN functional_parquet.alltypes b ON a.id = b.id;

Workaround, disable further min/max filter options to avoid initialising the overlap predicate:

SET MINMAX_FILTER_THRESHOLD = 0.0;
SET MINMAX_FILTER_SORTED_COLUMNS=FALSE;
SET MINMAX_FILTER_PARTITION_COLUMNS=FALSE;

Stack:

I0110 12:50:18.481575 12627 jni-util.cc:286] 1c493743f9081832:186a787f00000000] java.lang.NullPointerException
        at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:878)
        at org.apache.impala.planner.HdfsScanNode.initOverlapPredicate(HdfsScanNode.java:744)
        at org.apache.impala.planner.RuntimeFilterGenerator.assignRuntimeFilters(RuntimeFilterGenerator.java:925)
        at org.apache.impala.planner.RuntimeFilterGenerator.generateFilters(RuntimeFilterGenerator.java:835)
        at org.apache.impala.planner.RuntimeFilterGenerator.generateFilters(RuntimeFilterGenerator.java:838)
        at org.apache.impala.planner.RuntimeFilterGenerator.generateFilters(RuntimeFilterGenerator.java:828)
        at org.apache.impala.planner.RuntimeFilterGenerator.generateFilters(RuntimeFilterGenerator.java:838)
        at org.apache.impala.planner.RuntimeFilterGenerator.generateRuntimeFilters(RuntimeFilterGenerator.java:699)
        at org.apache.impala.planner.Planner.createPlanFragments(Planner.java:144)
        at org.apache.impala.planner.Planner.createPlans(Planner.java:249)
        at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1562)
        at org.apache.impala.service.Frontend.getPlannedExecRequest(Frontend.java:1939)
        at org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:1779)
        at org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:1644)
        at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1614)
        at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:164)

Issue Links

• relates to: IMPALA-11047 Preconditions.checkNotNull(statsTuple_) fail in HdfsScanNode.java if PARQUET_READ_STATISTICS=0