Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-16448

Vectorization: Vectorized order_null.q fails with deserialize EOF exception below TEZ ReduceRecordSource.processVectorGroup

Log workAgile BoardRank to TopRank to BottomBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • None
    • 3.0.0
    • Hive
    • None

    Description

      For query "SELECT x.* FROM src_null x ORDER BY b asc, a asc nulls last" here is the stack trace:

      ], TaskAttempt 1 failed, info=[Error: Error while running task ( failure ) : attempt_1492136345968_0001_40_01_000000_1:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: 
      DeserializeRead details: Reading inputByteBuffer of length 3 at start offset 0 for length 3 to read 2 fields with types [string, int].  Read field #1 at field start position 1 current read offset 3 column sort order [false, false]
      	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:211)
      	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:168)
      	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
      	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
      	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:422)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1807)
      	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
      	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
      	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
      	at org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: 
      DeserializeRead details: Reading inputByteBuffer of length 3 at start offset 0 for length 3 to read 2 fields with types [string, int].  Read field #1 at field start position 1 current read offset 3 column sort order [false, false]
      	at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecordVector(ReduceRecordSource.java:389)
      	at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:245)
      	at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:317)
      	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:185)
      	... 15 more
      Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: 
      DeserializeRead details: Reading inputByteBuffer of length 3 at start offset 0 for length 3 to read 2 fields with types [string, int].  Read field #1 at field start position 1 current read offset 3 column sort order [false, false]
      	at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.processVectorGroup(ReduceRecordSource.java:421)
      	at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecordVector(ReduceRecordSource.java:380)
      	... 18 more
      Caused by: java.io.EOFException
      	at org.apache.hadoop.hive.serde2.binarysortable.InputByteBuffer.read(InputByteBuffer.java:54)
      	at org.apache.hadoop.hive.serde2.binarysortable.fast.BinarySortableDeserializeRead.readNextField(BinarySortableDeserializeRead.java:205)
      	at org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow.deserialize(VectorDeserializeRow.java:751)
      	at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.processVectorGroup(ReduceRecordSource.java:417)
      	... 19 more
      

      Attachments

        1. HIVE-16448.01.patch
          75 kB
          Matt McCline
        2. HIVE-16448.02.patch
          104 kB
          Matt McCline
        3. HIVE-16448.03.patch
          149 kB
          Matt McCline

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            mmccline Matt McCline Assign to me
            mmccline Matt McCline
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment