[DRILL-2249] Parquet reader hit IOBE when reading decimal type columns.

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Component/s: Storage - Parquet, Storage - Writer
• Assignee: Steven Phillips
• Reporter: Jinfeng Ni
• Created: 14/Feb/15 00:03
• Updated: 08/May/15 20:50
• Resolved: 20/Feb/15 01:02

Description

On today's master branch:

select commit_id from sys.version;
------------

commit_id

------------

4ed0a8d68ec5ef344fb54ff7c9d857f7f3f153aa

------------

If I create a parquet file containing two decimal(10,2) columns as:

create table my_dec_table as select *, cast(o_totalprice as decimal(10,2)) dec1, cast(o_totalprice as decimal(10,2)) dec2 from cp.`tpch/orders.parquet`;

+------------+---------------------------+
| Fragment | Number of records written |
+------------+---------------------------+
| 0_0 | 15000 |
+------------+---------------------------+
1 row selected (1.977 seconds)

However, when I try to read from the new created parquet file, Drill report IOBE in parquet reader.

select * from my_dec_table;
Query failed: Query stopped., index: 22531, length: 1 (expected: range(0, 22531)) [ ee35bc67-5c70-4677-bf7f-8db12e4a5491 on 10.250.0.8:31010 ]

The plan looks fine to me for this query:

xplain plan for select * from my_dec_table;
+------------+------------+
|    text    |    json    |
+------------+------------+
| 00-00    Screen
00-01      Scan(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath [path=file:/Users/jni/work/data/tpcds/my_dec_table]], selectionRoot=/Users/jni/work/data/tpcds/my_dec_table, numFiles=1, columns=[`*`]]])

Here is part of the stack trace:

java.lang.IndexOutOfBoundsException: index: 22531, length: 1 (expected: range(0, 22531))
	at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:156) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
	at io.netty.buffer.DrillBuf.chk(DrillBuf.java:178) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
	at io.netty.buffer.DrillBuf.getByte(DrillBuf.java:673) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
	at org.apache.drill.exec.store.parquet.columnreaders.FixedByteAlignedReader$DateReader.readIntLittleEndian(FixedByteAlignedReader.java:144) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
	at org.apache.drill.exec.store.parquet.columnreaders.FixedByteAlignedReader.......

Issue Links

• is duplicated by: DRILL-2253 Vectorized Parquet reader fails to read correctly against RLE Dictionary encoded DATE column