[ORC-672] Fix made in ORC-598 need to be extended to other readers like DecimalFromFloatTreeReader

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.5.1, 1.6.1, 1.7.0
• Fix Version/s: 1.6.6, 1.7.0, 1.5.13
• Component/s: Java
• Assignee: Panagiotis Garefalakis
• Reporter: Taraka Rama Rao Lethavadla
• Created: 07/Oct/20 15:57
• Updated: 10/Aug/21 18:33
• Resolved: 09/Nov/20 22:08

Description

Scenario:
Orc file contains data(more than 1024 entries) in array<float> and an external table is created with one of the column as array<decimal(9,2)>. Now 

LOAD DATA INPATH '/tmp/test.orc' OVERWRITE INTO TABLE test_orc_table

query is failing with below exception

Caused by: java.lang.RuntimeException: java.lang.ArrayIndexOutOfBoundsException: 1024 
at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcInputFormat$VectorizedOrcRecordReader.next(VectorizedOrcInputFormat.java:133) at 
org.apache.hadoop.hive.ql.io.orc.VectorizedOrcInputFormat$VectorizedOrcRecordReader.next(VectorizedOrcInputFormat.java:56) at 
org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:361) ... 23 more Caused by: 
java.lang.ArrayIndexOutOfBoundsException: 1024 at org.apache.orc.impl.TreeReaderFactory$TreeReader.nextVector(TreeReaderFactory.java:270) at 
org.apache.orc.impl.TreeReaderFactory$FloatTreeReader.nextVector(TreeReaderFactory.java:667) at 
org.apache.orc.impl.ConvertTreeReaderFactory$DecimalFromFloatTreeReader.nextVector(ConvertTreeReaderFactory.java:1251) at 
org.apache.orc.impl.TreeReaderFactory$ListTreeReader.nextVector(TreeReaderFactory.java:2199) at 
org.apache.orc.impl.TreeReaderFactory$StructTreeReader.nextBatch(TreeReaderFactory.java:2013) at 
org.apache.orc.impl.RecordReaderImpl.nextBatch(RecordReaderImpl.java:1290) at 
org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.nextBatch(RecordReaderImpl.java:156) at 
org.apache.hadoop.hive.ql.io.orc.VectorizedOrcInputFormat$VectorizedOrcRecordReader.next(VectorizedOrcInputFormat.java:129) ... 25 more

So fix similar to ORC-598 can be done to other readers like DecimalFromFloatTreeReader

Issue Links

• relates to: ORC-598 Unable to read ORC file with struct and array.length > 1024
• links to: GitHub Pull Request #568
• links to: GitHub Pull Request #569