[ORC-598] Unable to read ORC file with struct and array.length > 1024

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.5.1, 1.6.2
• Fix Version/s: 1.6.6, 1.7.0, 1.5.13
• Component/s: Java
• Assignee: Krisztian Kasa
• Reporter: Krisztian Kasa
• Created: 12/Feb/20 18:30
• Updated: 10/Dec/20 23:36
• Resolved: 14/Feb/20 07:02

Description

Repro

Attached orc file schema

java -jar orc-1.5.5/java/tools/target/orc-tools-1.5.5-uber.jar meta bigarray.orc
log4j:WARN No appenders could be found for logger (org.apache.hadoop.util.Shell).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Processing data file bigarray.orc [length: 2337]
Structure for bigarray.orc
File Version: 0.12 with FUTURE
Rows: 1
Compression: ZLIB
Compression size: 262144
Type: struct<col1:array<decimal(5,1)>>

CREATE EXTERNAL TABLE t1(col1 array<decimal(6,1)>)
STORED AS orc
LOCATION '<location of containing folder of bigarray.orc>';

SELECT * FROM t1;

Error: java.io.IOException: java.lang.ArrayIndexOutOfBoundsException: 1024 (state=,code=0)

Issue Links

• is related to: ORC-672 Fix made in ORC-598 need to be extended to other readers like DecimalFromFloatTreeReader
• links to: GitHub Pull Request #479