[HIVE-12208] Vectorized JOIN NPE on dynamically partitioned hash-join + map-join

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 1.3.0, 2.0.0
• Component/s: Vectorization
• Assignee: Gunther Hagleitner
• Reporter: Gopal Vijayaraghavan
• Created: 18/Oct/15 07:10
• Updated: 16/Feb/16 23:52
• Resolved: 11/Nov/15 21:48

Description

TPC-DS Q82 with reducer vectorized join optimizations

  Reducer 5 <- Map 1 (CUSTOM_SIMPLE_EDGE), Map 2 (CUSTOM_SIMPLE_EDGE), Map 3 (BROADCAST_EDGE), Map 4 (CUSTOM_SIMPLE_EDGE)

set hive.optimize.dynamic.partition.hashjoin=true;
set hive.vectorized.execution.reduce.enabled=true;
set hive.mapjoin.hybridgrace.hashtable=false;

select  i_item_id
       ,i_item_desc
       ,i_current_price
 from item, inventory, date_dim, store_sales
 where i_current_price between 30 and 30+30
 and inv_item_sk = i_item_sk
 and d_date_sk=inv_date_sk
 and d_date between '2002-05-30' and '2002-07-30'
 and i_manufact_id in (437,129,727,663)
 and inv_quantity_on_hand between 100 and 500
 and ss_item_sk = i_item_sk
 group by i_item_id,i_item_desc,i_current_price
 order by i_item_id
 limit 100

possibly a trivial plan setup issue, since the NPE is pretty much immediate.

Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException
	at org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerLongOperator.process(VectorMapJoinInnerLongOperator.java:368)
	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:852)
	at org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinGenerateResultOperator.forwardBigTableBatch(VectorMapJoinGenerateResultOperator.java:603)
	at org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerLongOperator.process(VectorMapJoinInnerLongOperator.java:362)
	... 19 more
Caused by: java.lang.NullPointerException
	at org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerGenerateResultOperator.commonSetup(VectorMapJoinInnerGenerateResultOperator.java:112)
	at org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerLongOperator.process(VectorMapJoinInnerLongOperator.java:96)
	... 22 more