Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-18413

Grouping of an empty result set may only contain null values

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 3.0.0
    • None
    • None

    Description

      exposed by: HIVE-18359

      in case of vectorization, the summary row object was left as is (presumed null earlier); which may cause it to be inconsistent isNull conditions in .VectorHashKeyWrapperBatch

      issue happens only if:

      • vectorizable groupby
      • groupping set contains empty
      • non-trivial empty; mapper is run
      • groupping key is select ; with a type which is backed by a bytea; ex:string
      set hive.vectorized.execution.enabled=true;
      create table tx2 (a integer,b integer,c integer,d double,u string,bi binary) stored as orc;
      
      insert into tx2 values
      (1,2,3,1.1,'x','b'),
      (3,2,3,1.1,'y','b');
      
      select  sum(a),
              u,
              bi,
              'asd',
              grouping(bi),
              'NULL,1' as expected
      from    tx2
      where   a=2
      group by a,u,bi grouping sets ( u, (), bi);
      

      causes:

      Caused by: java.lang.NullPointerException
              at java.lang.System.arraycopy(Native Method)
              at org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:173)
              at org.apache.hadoop.hive.ql.exec.vector.VectorHashKeyWrapperBatch.assignRowColumn(VectorHashKeyWrapperBatch.java:1065)
              at org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.writeSingleRow(VectorGroupByOperator.java:1134)
              at org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.access$800(VectorGroupByOperator.java:74)
              at org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeReduceMergePartial.close(VectorGroupByOperator.java:862)
              at org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.closeOp(VectorGroupByOperator.java:1176)
              at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:705)
              at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.close(ReduceRecordProcessor.java:383)
              ... 16 more
      ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1515531021543_0001_12_01 [Reducer 2] killed/failed due to:OWN_TASK_FAILURE]DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:0
      2018-01-09T12:50:30,611 DEBUG [01fdcefd-40b0-45a6-8e5b-b1cd14241088 main] ql.Driver: Shutting down query 
      

      Attachments

        1. HIVE-18413.03.patch
          10 kB
          Zoltan Haindrich
        2. HIVE-18413.02.patch
          7 kB
          Zoltan Haindrich
        3. HIVE-18413.01.patch
          4 kB
          Zoltan Haindrich

        Issue Links

          Activity

            People

              kgyrtkirk Zoltan Haindrich
              kgyrtkirk Zoltan Haindrich
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: