[HIVE-18114] Fix LazySimpleDeserializeRead failed precondition

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Zoltan Haindrich
• Created: 21/Nov/17 15:52
• Updated: 21/Nov/17 15:52

Description

Exception:

[...]
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row 
        at org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:919)
        at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.processRow(MapRecordSource.java:92)
        ... 18 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: 
DeserializeRead detail: Reading byte[] of length 142 at start offset 0 for length 42 to read 3 fields wi
th types [int, string, struct<c1:boolean,c2:tinyint,c3:smallint,c4:int,c5:bigint,c6:float,c7:double,c8:d
ecimal(38,18),c9:char(25),c10:varchar(25),c11:timestamp,c12:date,c13:binary>].  Read field #2 at field s
tart position 11 for field length 31
        at org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:881)
        ... 19 more
Caused by: java.lang.IllegalStateException
        at com.google.common.base.Preconditions.checkState(Preconditions.java:133)
        at org.apache.hadoop.hive.serde2.lazy.fast.LazySimpleDeserializeRead.readComplexField(LazySimple
DeserializeRead.java:1060)
        at org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow.storeComplexFieldRowColumn(VectorD
eserializeRow.java:748)
        at org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow.storeStructRowColumn(VectorDeseria
lizeRow.java:856)
        at org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow.storeRowColumn(VectorDeserializeRo
w.java:919)
        at org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow.deserialize(VectorDeserializeRow.j
ava:1332)
        at org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:876)
        ... 19 more

reproduce:

# apply attached patch (changes qtest)
mvn install  -Pitests -pl itests/qtest -Dtest=Test*CliDriver#*[schema_evol_text_vec_part_all_complex]   -DskipSparkTests -q -DinitScript=asd.sql -Dtest.output.overwrite  -am