[HIVE-10011] LLAP: NegativeArraySize exception on vector string reader

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Fix Version/s: llap
• Assignee: Sergey Shelukhin Assign to me
• Reporter: Gopal Vijayaraghavan
• Created: 19/Mar/15 01:50
• Updated: 17/Feb/16 04:33
• Resolved: 19/Mar/15 19:35

Description

With some logging, I confirmed that the String length vectors contained junk data & the length field is overflowing.

Caused by: java.lang.NegativeArraySizeException
    at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$BytesColumnVectorUtil.commonReadByteArrays(RecordReaderImpl.java:1550)
    at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$BytesColumnVectorUtil.readOrcByteArrays(RecordReaderImpl.java:1569)
    at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$StringDirectTreeReader.nextVector(RecordReaderImpl.java:1691)
    at org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl$StringTreeReader.nextVector(RecordReaderImpl.java:1517)
    at org.apache.hadoop.hive.llap.io.decode.OrcEncodedDataConsumer.decodeBatch(OrcEncodedDataConsumer.java:115)
    at org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer.consumeData(EncodedDataConsumer.java:108)
    at org.apache.hadoop.hive.llap.io.decode.EncodedDataConsumer.consumeData(EncodedDataConsumer.java:35)
    at org.apache.hadoop.hive.ql.io.orc.EncodedReaderImpl.readEncodedColumns(EncodedReaderImpl.java:314)
    at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:272)
    at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:44)
    at org.apache.hadoop.hive.common.CallableWithNdc.call(CallableWithNdc.java:37)
    ... 4 more

Issue Links

• duplicates: HIVE-9979 LLAP: LLAP Cached readers for StringDirectTreeReaders over-read data