Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-6137

Join Failure When Some Json File Partitions Empty

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.13.0
    • Fix Version/s: 1.13.0
    • Component/s: None
    • Labels:
      None

      Description

      The following exception can occurr when the following query is executed:

      select t.p_partkey, t1.ps_suppkey from dfs.`join/empty_part/part` as t RIGHT JOIN dfs.`join/empty_part/partsupp` as t1 ON t.p_partkey = t1.ps_partkey where t1.ps_partkey > 1
      
      • part has one nonempty file 0_0_0.json
      • partsupp has one nonempty file 0_0_0.json and one empty file 0_0_1.json
        (java.lang.IllegalStateException) next() [on #10, RemovingRecordBatch] called again after it returned NONE.  Caller should not have called next() again.
          org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():220
          org.apache.drill.exec.record.AbstractRecordBatch.next():119
          org.apache.drill.exec.test.generated.HashJoinProbeGen2.executeProbePhase():119
          org.apache.drill.exec.test.generated.HashJoinProbeGen2.probeAndProject():227
          org.apache.drill.exec.physical.impl.join.HashJoinBatch.innerNext():222
          org.apache.drill.exec.record.AbstractRecordBatch.next():164
          org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():228
          org.apache.drill.exec.record.AbstractRecordBatch.next():119
          org.apache.drill.exec.record.AbstractRecordBatch.next():109
          org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51
          org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():134
          org.apache.drill.exec.record.AbstractRecordBatch.next():164
          org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():228
          org.apache.drill.exec.record.AbstractRecordBatch.next():119
          org.apache.drill.exec.record.AbstractRecordBatch.next():109
          org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51
          org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():134
          org.apache.drill.exec.record.AbstractRecordBatch.next():164
          org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():228
          org.apache.drill.exec.physical.impl.BaseRootExec.next():105
          org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext():93
          org.apache.drill.exec.physical.impl.BaseRootExec.next():95
          org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():233
          org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():226
          java.security.AccessController.doPrivileged():-2
          javax.security.auth.Subject.doAs():415
          org.apache.hadoop.security.UserGroupInformation.doAs():1657
          org.apache.drill.exec.work.fragment.FragmentExecutor.run():226
          org.apache.drill.common.SelfCleaningRunnable.run():38
          java.util.concurrent.ThreadPoolExecutor.runWorker():1145
          java.util.concurrent.ThreadPoolExecutor$Worker.run():615
          java.lang.Thread.run():745
      

        Attachments

          Activity

            People

            • Assignee:
              timothyfarkas Timothy Farkas
              Reporter:
              timothyfarkas Timothy Farkas
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: