[HIVE-15942] Q22 does not get vectorized due to grouping set evaluations

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Assignee: Unassigned
• Reporter: Rajesh Balamohan
• Created: 16/Feb/17 12:16
• Updated: 04/May/17 06:12
• Resolved: 04/May/17 06:12

Description

Env: apache hive master with LLAP + tez master
Query: q22 @ tpcds 10 TB scale

Map-1 does not get vectorized. Hive logs point out that group set evaluation is preventing it from vectorization

2017-02-16T07:10:06,074  INFO [c9d014ef-5a60-4ef1-b7a8-5209da679ebf main] physical.Vectorizer: ReduceWorkVectorizationNodeProcessor process reduceColumnNames [VALUE._col0, VALUE._col1, VALUE._col2]
2017-02-16T07:10:06,074  INFO [c9d014ef-5a60-4ef1-b7a8-5209da679ebf main] physical.Vectorizer: ReduceWorkVectorizationNodeProcessor process operator GBY using vectorization contextContext name __Reduce_Shuffle__, level 0, sorted projectionColumnMap {0=VALUE._col0, 1=VALUE._col1, 2=VALUE._col2}, scratchColumnTypeNames []
2017-02-16T07:10:06,074  INFO [c9d014ef-5a60-4ef1-b7a8-5209da679ebf main] physical.Vectorizer: ReduceWorkVectorizationNodeProcessor process going to walk the operator stack to get vectorization context for RS
2017-02-16T07:10:06,075  INFO [c9d014ef-5a60-4ef1-b7a8-5209da679ebf main] physical.Vectorizer: walkStackToFindVectorizationContext GBY has new vectorization context Context name GBY, level 0, sorted projectionColumnMap {0=_col0, 1=_col1, 2=_col2}, scratchColumnTypeNames []
2017-02-16T07:10:06,075  INFO [c9d014ef-5a60-4ef1-b7a8-5209da679ebf main] physical.Vectorizer: ReduceWorkVectorizationNodeProcessor process operator RS using vectorization contextContext name GBY, level 0, sorted projectionColumnMap {0=_col0, 1=_col1, 2=_col2}, scratchColumnTypeNames []
2017-02-16T07:10:06,075  INFO [c9d014ef-5a60-4ef1-b7a8-5209da679ebf main] physical.Vectorizer: Validating MapWork...
2017-02-16T07:10:06,084  INFO [c9d014ef-5a60-4ef1-b7a8-5209da679ebf main] physical.Vectorizer: Cannot vectorize: GROUPBY operator: Grouping sets not supported
2017-02-16T07:10:06,084  INFO [c9d014ef-5a60-4ef1-b7a8-5209da679ebf main] physical.Vectorizer: Validating ReduceWork...
2017-02-16T07:10:06,084  INFO [c9d014ef-5a60-4ef1-b7a8-5209da679ebf main] physical.Vectorizer: Cannot vectorize: GROUPBY operator: Pruning grouping set id not supported
2017-02-16T07:10:06,085  INFO [c9d014ef-5a60-4ef1-b7a8-5209da679ebf main] physical.Vectorizer: Validating ReduceWork...
2017-02-16T07:10:06,086  INFO [c9d014ef-5a60-4ef1-b7a8-5209da679ebf main] physical.Vectorizer: Vectorizing ReduceWork...
2017-02-16T07:10:06,086  INFO [c9d014ef-5a60-4ef1-b7a8-5209da679ebf main] physical.Vectorizer: vectorizeReduceWork reducer Operator: SEL...
2017-02-16T07:10:06,086  INFO [c9d014ef-5a60-4ef1-b7a8-5209da679ebf main] physical.Vectorizer: ReduceWorkVectorizationNodeProcessor process reduceColumnNames [KEY.reducesinkkey0, KEY.reducesinkkey1, KEY.reducesinkkey2, KEY.reducesinkkey3, KEY.reducesinkkey4]
2017-02-16T07:10:06,086  INFO [c9d014ef-5a60-4ef1-b7a8-5209da679ebf main] physical.Vectorizer: ReduceWorkVectorizationNodeProcessor process operator SEL using vectorization contextContext name __Reduce_Shuffle__, level 0, sorted projectionColumnMap {0=KEY.reducesinkkey0, 1=KEY.reducesinkkey1, 2=KEY.reducesinkkey2, 3=KEY.reducesinkkey3, 4=KEY.reducesinkkey4}, scratchColumnTypeNames []

Issue Links

• is blocked by: HIVE-14016 Vectorization: Add support for Grouping Sets