[DRILL-6694] NPE in UnnestRecordBatch when query uses a column name not present in data

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.14.0
• Fix Version/s: 1.15.0
• Component/s: Execution - Relational Operators
• Labels: ready-to-commit
• Assignee: Sorabh Hamirwasia
• Reporter: Sorabh Hamirwasia
• Reviewer: Parth Chandra
• Created: 16/Aug/18 21:55
• Updated: 17/Aug/18 21:10
• Resolved: 17/Aug/18 21:10

Description

When the array column name doesn't exist in the underlying data and is used in query with Unnest then there is NPE. The reason is Unnest tries to get the ValueVector of unnest column from incoming based on TypedFieldId which will be null in this case and hence the exception.

[Error Id: 6f8461ee-92c7-4865-b5e6-3e2f756391c4 on pssc-67.qa.lab:31010] at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633) ~[drill-common-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:361) [drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:216) [drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:327) [drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) [drill-common-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_151] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_151] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_151] Caused by: java.lang.NullPointerException: null at org.apache.drill.exec.physical.impl.unnest.UnnestRecordBatch.schemaChanged(UnnestRecordBatch.java:422) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.physical.impl.unnest.UnnestRecordBatch.innerNext(UnnestRecordBatch.java:208) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:172) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:64) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:142) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:172) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.physical.impl.join.LateralJoinBatch.prefetchFirstBatchFromBothSides(LateralJoinBatch.java:331) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.physical.impl.join.LateralJoinBatch.buildSchema(LateralJoinBatch.java:356) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:152) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.physical.impl.aggregate.StreamingAggBatch.buildSchema(StreamingAggBatch.java:158) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:152) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:103) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext(SingleSenderCreator.java:93) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:93) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:294) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:281) ~[drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_151] at javax.security.auth.Subject.doAs(Subject.java:422) ~[na:1.8.0_151] at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1633) ~[hadoop-common-2.7.0-mapr-1707.jar:na] at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:281) [drill-java-exec-1.15.0-SNAPSHOT.jar:1.15.0-SNAPSHOT] ... 4 common frames omitted 14:18:13.952 [qtp667758892-38873] ERROR o.a.d.e.server.rest.QueryResources - Query from Web UI Failed org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: NullPointerException

 

Query:

 select count(t.l.l_shipdate) from dfs.nestedPar.`cl60_60` c, unnest(c.c_ldata) t(l)

 

Issue Links

• links to: GitHub Pull Request #1434