[PARQUET-244] DeltaByteArrayReader fails with ArrayIndexOutOfBoundsException when moving across pages

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.6.0
• Component/s: parquet-mr
• Assignee: Unassigned
• Reporter: Alosh Bennett
• Created: 07/Apr/15 12:22
• Updated: 17/Jun/15 16:18
• Resolved: 17/Jun/15 16:18

Description

DeltaByteArrayReader.readBytes() fails with ArrayIndexOutOfBoundsException soon after it has processed a new page via initFromPage(). This issue can be reproduced by trying to read a Binary column that is encoded using delta byte array and spans multiple pages.

This is happening because ColumnReaderImpl.initDataReader() creates a new ValueReader every time a new page is processed (see this.dataColumn = dataEncoding.getValuesReader(path, VALUES)). The DeltaByteArrayReader is stateful and needs to remember the previous Binary value that was read across pages. When a new DeltaByteArrayReader is created, this information is lost.

Issue Links

• is duplicated by: PARQUET-246 ArrayIndexOutOfBoundsException with Parquet write version v2