[DRILL-1314] Parquet Reader for compressed parquet files fails

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.5.0
• Fix Version/s: 0.5.0
• Component/s: Storage - Parquet
• Labels: Parquet Storage

Environment

10+1 Nodes; RHEL 6.4; 256GB RAM/node; 32 CPU cores/node

• Assignee: Jason Altekruse
• Reporter: Kunal Khatua
• Created: 19/Aug/14 06:03
• Updated: 20/May/15 23:27
• Resolved: 31/Aug/14 17:40

Description

When querying compressed Parquet files, an error is reported for the RLE stream:
select
l_orderkey, l_partkey, l_suppkey, l_linenumber, l_quantity, l_extendedprice, l_discount, l_tax, l_returnflag, l_linestatus, l_shipdate, l_commitdate, l_receiptdate, l_shipinstruct, l_shipmode, l_comment
from
lineitem_imp100
where
l_orderkey > 0
and l_partkey >= 0
and l_suppkey >= 0
and l_linenumber >= 0
and l_quantity >= 0
and l_extendedprice >= 0
and l_discount >= 0
and l_tax >= 0
and length(l_returnflag) > 0
and length(l_linestatus) > 0
and length(l_shipdate) > 0
and length(l_commitdate) > 0
and length(l_receiptdate) > 0
and length(l_shipinstruct) > 0
and length(l_shipmode) > length(l_comment);
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

l_orderkey

l_partkey

l_suppkey

l_linenumber

l_quantity

l_extendedprice

l_discount

l_tax

l_returnflag

l_linestatus

l_shipdate

l_commitdate

l_receiptdate

l_shipinstruct

l_shipmode

l_comment

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Query failed: Failure while running fragment. Reading past RLE/BitPacking stream. [3a999343-a881-4193-9853-27e24aaf768b]
java.lang.RuntimeException: java.sql.SQLException: Failure while trying to get next result batch.
at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
at sqlline.SqlLine.print(SqlLine.java:1809)
at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
at sqlline.SqlLine.dispatch(SqlLine.java:889)
at sqlline.SqlLine.begin(SqlLine.java:763)
at sqlline.SqlLine.start(SqlLine.java:498)
at sqlline.SqlLine.main(SqlLine.java:460)

The JDBC client reports the stack trace:
java.sql.SQLException: exception while executing query: Failure while trying to get next result batch.
at net.hydromatic.avatica.Helper.createException(Helper.java:40)
at net.hydromatic.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:406)
at net.hydromatic.avatica.AvaticaStatement.executeQueryInternal(AvaticaStatement.java:351)
at net.hydromatic.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:78)
at PipSQueak.executeQuery(PipSQueak.java:243)
at PipSQueak.runTest(PipSQueak.java:81)
at PipSQueak.main(PipSQueak.java:404)
Caused by: java.sql.SQLException: Failure while trying to get next result batch.
at org.apache.drill.jdbc.DrillCursor.next(DrillCursor.java:110)
at org.apache.drill.jdbc.DrillResultSet.execute(DrillResultSet.java:90)
at org.apache.drill.jdbc.DrillResultSet.execute(DrillResultSet.java:44)
at net.hydromatic.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:404)
... 5 more
Caused by: org.apache.drill.exec.rpc.RpcException: Failure while running fragment. Reading past RLE/BitPacking stream. [2cc52105-15ac-49ba-a93b-37cf8d47e3c8]

at org.apache.drill.exec.rpc.user.QueryResultHandler.batchArrived(QueryResultHandler.java:77)
at org.apache.drill.exec.rpc.user.UserClient.handleReponse(UserClient.java:90)
at org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:52)
at org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:34)
at org.apache.drill.exec.rpc.RpcBus.handle(RpcBus.java:60)
at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:181)
at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:165)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:318)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:318)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:163)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:318)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:318)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:125)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:507)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:464)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:378)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:350)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
at java.lang.Thread.run(Thread.java:744)