[DRILL-4165] IllegalStateException in MergeJoin for a query against TPC-DS data

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.4.0
• Fix Version/s: 1.4.0
• Component/s: Execution - Relational Operators
• Assignee: amit hadke
• Reporter: Aman Sinha
• Reviewer: Victoria Markman
• Created: 07/Dec/15 03:21
• Updated: 15/Dec/15 19:32
• Resolved: 08/Dec/15 08:29

Description

I am seeing the following on the 1.4.0 branch.

0: jdbc:drill:zk=local> alter session set `planner.enable_hashjoin` = false;
..
0: jdbc:drill:zk=local> select count(*) from dfs.`tpcds/store_sales` ss1, dfs.`tpcds/store_sales` ss2 where ss1.ss_customer_sk = ss2.ss_customer_sk and ss1.ss_store_sk = 1 and ss2.ss_store_sk = 2;
Error: SYSTEM ERROR: IllegalStateException: Incoming batch [#55, MergeJoinBatch] has size 1984616, which is beyond the limit of 65536

Fragment 0:0

[Error Id: 18bf00fe-52d7-4d84-97ec-b04a035afb4e on 192.168.1.103:31010]

  (java.lang.IllegalStateException) Incoming batch [#55, MergeJoinBatch] has size 1984616, which is beyond the limit of 65536
    org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():305
    org.apache.drill.exec.record.AbstractRecordBatch.next():119
    org.apache.drill.exec.record.AbstractRecordBatch.next():109
    org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51
    org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():132