[DRILL-7865] Unable to apply DrillPushLimitToScanRule due to NPE

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.19.0
• Assignee: Vova Vysotskyi
• Reporter: Vova Vysotskyi
• Created: 19/Feb/21 20:56
• Updated: 20/Feb/21 10:54
• Resolved: 20/Feb/21 10:54

Description

At some point of planning, DrillPushLimitToScanRule fails with NPE, here is the stack trace:

WARN  o.a.d.e.p.l.DrillPushLimitToScanRule - Exception while using the pruned partitions.
java.lang.NullPointerException: null
	at org.apache.drill.exec.store.dfs.easy.EasyGroupScan.initFromSelection(EasyGroupScan.java:192)
	at org.apache.drill.exec.store.dfs.easy.EasyGroupScan.access$1300(EasyGroupScan.java:77)
	at org.apache.drill.exec.store.dfs.easy.EasyGroupScan$EasyGroupScanFilterer.build(EasyGroupScan.java:414)
	at org.apache.drill.exec.physical.base.AbstractGroupScanWithMetadata.applyLimit(AbstractGroupScanWithMetadata.java:470)
	at org.apache.drill.exec.planner.logical.DrillPushLimitToScanRule.doOnMatch(DrillPushLimitToScanRule.java:98)
	at org.apache.drill.exec.planner.logical.DrillPushLimitToScanRule$1.onMatch(DrillPushLimitToScanRule.java:56)
	at org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:319)
	at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:561)
	at org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:420)
	at org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:257)
	at org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:127)
	at org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:216)
	at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:203)
	at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform(DefaultSqlHandler.java:394)
	at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform(DefaultSqlHandler.java:351)
	at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.transform(DefaultSqlHandler.java:338)
	at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRawDrel(DefaultSqlHandler.java:252)
	at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:308)
	at org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:173)
	at org.apache.drill.exec.planner.sql.DrillSqlWorker.getQueryPlan(DrillSqlWorker.java:283)
	at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPhysicalPlan(DrillSqlWorker.java:163)
	at org.apache.drill.exec.planner.sql.DrillSqlWorker.convertPlan(DrillSqlWorker.java:128)
	at org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:93)
	at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:593)
	at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:274)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

This stack trace may be observed for example for TestImageRecordReader.testLimitPushdown() test.
Though the query is not failed, it may prevent applying limit pushdown for some cases.