[HIVE-18413] Grouping of an empty result set may only contain null values

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0
• Assignee: Zoltan Haindrich
• Reporter: Zoltan Haindrich
• Created: 09/Jan/18 11:21
• Updated: 22/May/18 23:12
• Resolved: 18/Jan/18 18:22

Description

exposed by: HIVE-18359

in case of vectorization, the summary row object was left as is (presumed null earlier); which may cause it to be inconsistent isNull conditions in .VectorHashKeyWrapperBatch

issue happens only if:

• vectorizable groupby
• groupping set contains empty
• non-trivial empty; mapper is run
• groupping key is select ; with a type which is backed by a bytea; ex:string

set hive.vectorized.execution.enabled=true;
create table tx2 (a integer,b integer,c integer,d double,u string,bi binary) stored as orc;

insert into tx2 values
(1,2,3,1.1,'x','b'),
(3,2,3,1.1,'y','b');

select  sum(a),
        u,
        bi,
        'asd',
        grouping(bi),
        'NULL,1' as expected
from    tx2
where   a=2
group by a,u,bi grouping sets ( u, (), bi);

causes:

Caused by: java.lang.NullPointerException
        at java.lang.System.arraycopy(Native Method)
        at org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:173)
        at org.apache.hadoop.hive.ql.exec.vector.VectorHashKeyWrapperBatch.assignRowColumn(VectorHashKeyWrapperBatch.java:1065)
        at org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.writeSingleRow(VectorGroupByOperator.java:1134)
        at org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.access$800(VectorGroupByOperator.java:74)
        at org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeReduceMergePartial.close(VectorGroupByOperator.java:862)
        at org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.closeOp(VectorGroupByOperator.java:1176)
        at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:705)
        at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:383)
        ... 16 more
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1515531021543_0001_12_01 [Reducer 2] killed/failed due to:OWN_TASK_FAILURE]DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
2018-01-09T12:50:30,611 DEBUG [01fdcefd-40b0-45a6-8e5b-b1cd14241088 main] ql.Driver: Shutting down query 

Issue Links

• is caused by: HIVE-17617 Rollup of an empty resultset should contain the grouping of the empty grouping set