Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-7091

Query with EXISTS and correlated subquery fails with NPE in HashJoinMemoryCalculatorImpl$BuildSidePartitioningImpl

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: In Progress
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.15.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Steps to reproduce:
      1. Create view:

      create view dfs.tmp.nation_view as select * from cp.`tpch/nation.parquet`;
      

      Run the following query:

      SELECT n_nationkey, n_name
      FROM  dfs.tmp.nation_view a
      WHERE EXISTS (SELECT 1
      FROM cp.`tpch/region.parquet` b
      WHERE b.r_regionkey =  a.n_regionkey)
      

      This query fails with NPE:

      [Error Id: 9a592635-f792-4403-965c-bd2eece7e8fc on cv1:31010]
      	at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633) ~[drill-common-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:364) [drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:219) [drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:330) [drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) [drill-common-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_161]
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_161]
      	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]
      Caused by: java.lang.NullPointerException: null
      	at org.apache.drill.exec.physical.impl.join.HashJoinMemoryCalculatorImpl$BuildSidePartitioningImpl.initialize(HashJoinMemoryCalculatorImpl.java:267) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.physical.impl.join.HashJoinBatch.executeBuildPhase(HashJoinBatch.java:959) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.physical.impl.join.HashJoinBatch.innerNext(HashJoinBatch.java:525) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:126) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:116) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:63) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:141) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:126) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.test.generated.HashAggregatorGen2.doWork(HashAggTemplate.java:642) ~[na:na]
      	at org.apache.drill.exec.physical.impl.aggregate.HashAggBatch.innerNext(HashAggBatch.java:295) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:126) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:116) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:63) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:141) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:126) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.physical.impl.join.HashJoinBatch.sniffNonEmptyBatch(HashJoinBatch.java:466) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.physical.impl.join.HashJoinBatch.prefetchFirstBatch(HashJoinBatch.java:429) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.physical.impl.join.HashJoinBatch.prefetchFirstBuildBatch(HashJoinBatch.java:377) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.physical.impl.join.HashJoinBatch.innerNext(HashJoinBatch.java:511) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:126) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:116) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:63) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:126) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:116) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:63) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:126) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:116) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:63) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:141) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:126) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:116) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:63) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:141) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:186) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:83) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:297) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:284) ~[drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_161]
      	at javax.security.auth.Subject.doAs(Subject.java:422) ~[na:1.8.0_161]
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1669) ~[hadoop-common-2.7.0-mapr-1808.jar:na]
      	at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:284) [drill-java-exec-1.16.0-SNAPSHOT.jar:1.16.0-SNAPSHOT]
      	... 4 common frames omitted
      2019-03-11 07:41:33,706 [23798c65-610f-2f27-93c8-7d07eb443380:frag:0:0] WARN  o.a.d.exec.rpc.control.WorkEventBus - Fragment 23798c65-610f-2f27-93c8-7d07eb443380:0:0 manager is not found in the work bus.
      2019-03-11 07:41:33,713 [qtp386796108-234] ERROR o.a.d.e.server.rest.QueryResources - Query from Web UI Failed: {}
      org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: NullPointerException
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                volodymyr Vova Vysotskyi
                Reviewer:
                Vova Vysotskyi
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: