[PARQUET-246] ArrayIndexOutOfBoundsException with Parquet write version v2

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.6.0
• Fix Version/s: 1.8.0
• Assignee: Konstantin Shaposhnikov
• Reporter: Konstantin Shaposhnikov
• Created: 08/Apr/15 07:49
• Updated: 09/Jul/15 17:52
• Resolved: 09/Jul/15 17:36

Description

I am getting the following exception when reading a parquet file that was created using Avro WriteSupport and Parquet write version v2.0:

Caused by: parquet.io.ParquetDecodingException: Can't read value in column [colName, rows, array, name] BINARY at value 313601 out of 428260, 1 out of 39200 in currentPage. repetition level: 0, definition level: 2
	at parquet.column.impl.ColumnReaderImpl.readValue(ColumnReaderImpl.java:462)
	at parquet.column.impl.ColumnReaderImpl.writeCurrentValueToConverter(ColumnReaderImpl.java:364)
	at parquet.io.RecordReaderImplementation.read(RecordReaderImplementation.java:405)
	at parquet.hadoop.InternalParquetRecordReader.nextKeyValue(InternalParquetRecordReader.java:209)
	... 27 more
Caused by: java.lang.ArrayIndexOutOfBoundsException
	at parquet.column.values.deltastrings.DeltaByteArrayReader.readBytes(DeltaByteArrayReader.java:70)
	at parquet.column.impl.ColumnReaderImpl$2$6.read(ColumnReaderImpl.java:307)
	at parquet.column.impl.ColumnReaderImpl.readValue(ColumnReaderImpl.java:458)
	... 30 more

The file is quite big (500Mb) so I cannot upload it here, but possibly there is enough information in the exception message to understand the cause of error.

Issue Links

• blocks: PARQUET-292 Release Parquet 1.8.0
• duplicates: PARQUET-244 DeltaByteArrayReader fails with ArrayIndexOutOfBoundsException when moving across pages
• links to: PR #171: Write-side fix for this bug.
• links to: PR #235: Correctly read data written with this bug.