[DRILL-2107] Hash Join throw IOBE for a query with exists subquery.

• Type: New Feature
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 0.8.0
• Component/s: Execution - Relational Operators
• Assignee: Mehant Baid
• Reporter: Jinfeng Ni
• Created: 29/Jan/15 01:05
• Updated: 12/Feb/15 23:00
• Resolved: 12/Feb/15 23:00

Description

I hit an IOBE for TestTpchDistributed Q4, when I tried to enable an optimizer rule. Then, I simplified Q4 to the following, and still re-produce the same IOBE.

select
  o.o_orderpriority
from
  cp.`tpch/orders.parquet` o
where
  exists (
    select
      *
    from
      cp.`tpch/lineitem.parquet` l
    where
      l.l_orderkey = o.o_orderkey
  )
;

Stack trace of the exception:

java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
     at java.util.ArrayList.rangeCheck(ArrayList.java:635) ~[na:1.7.0_45]
     at java.util.ArrayList.get(ArrayList.java:411) ~[na:1.7.0_45]
     at org.apache.drill.exec.record.VectorContainer.getValueAccessorById(VectorContainer.java:232) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
     at org.apache.drill.exec.record.RecordBatchLoader.getValueAccessorById(RecordBatchLoader.java:149) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
     at org.apache.drill.exec.physical.impl.unorderedreceiver.UnorderedReceiverBatch.getValueAccessorById(UnorderedReceiverBatch.java:132) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
     at org.apache.drill.exec.test.generated.HashTableGen307.doSetup(HashTableTemplate.java:71) ~[na:na]
     at org.apache.drill.exec.test.generated.HashTableGen307.updateBatches(HashTableTemplate.java:473) ~[na:na]
     at org.apache.drill.exec.test.generated.HashJoinProbeGen313.executeProbePhase(HashJoinProbeTemplate.java:139) ~[na:na]
     at org.apache.drill.exec.test.generated.HashJoinProbeGen313.probeAndProject(HashJoinProbeTemplate.java:223) ~[na:na]
     at org.apache.drill.exec.physical.impl.join.HashJoinBatch.innerNext(HashJoinBatch.java:227) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
....

The physical plan seems to be correct, after enabling the new rule. Actually, if I disable HashJoin, and use merge join for the query, it works fine. So, seems the IOBE exposes some bug in HashJoin.

To re-produce this issue, two options:
1 ) - Modify DrillRuleSets.java, remove the comment before SwapJoinRule

• alter session set `planner.slice_target` = 10;
• run the query

2) use the attached physical plan in json file, and use "submitplan" to submit the physical plan.

For comparison, I also attached the physical plan when disabling hashjoin (use merge join), and the explain plan at physical operator level.