[DRILL-496] drill stuck in loop when query parquet file and eventually ran out of memory

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.4.0
• Fix Version/s: 0.4.0
• Assignee: Steven Phillips
• Reporter: Chun Chang
• Created: 09/Apr/14 20:48
• Updated: 31/Jul/14 06:33
• Resolved: 20/Apr/14 04:13

Description

Ran the following Mondrian query against parquet format file and hit the issue that drill bit in infinite loop, and eventually run out of memory.

0: jdbc:drill:schema=dfs> select * from days;

drillbit log:

Exception in thread "WorkManager-3" java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:658)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306)
at io.netty.buffer.PoolArenaL$DirectArena.newUnpooledChunk(PoolArenaL.java:387)
at io.netty.buffer.PoolArenaL.allocateHuge(PoolArenaL.java:152)
at io.netty.buffer.PoolArenaL.allocate(PoolArenaL.java:129)
at io.netty.buffer.PoolArenaL.reallocate(PoolArenaL.java:234)
at io.netty.buffer.PooledByteBufL.capacity(PooledByteBufL.java:116)
at io.netty.buffer.AbstractByteBuf.ensureWritable(AbstractByteBuf.java:250)
at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:807)
at org.apache.drill.exec.memory.AccountingByteBuf.writeBytes(AccountingByteBuf.java:664)
at org.apache.drill.exec.store.parquet.NullableFixedByteAlignedReader.readField(NullableFixedByteAlignedReader.java:48)
at org.apache.drill.exec.store.parquet.NullableColumnReader.readAllFixedFields(NullableColumnReader.java:106)
at org.apache.drill.exec.store.parquet.ParquetRecordReader.readAllFixedFields(ParquetRecordReader.java:327)
at org.apache.drill.exec.store.parquet.ParquetRecordReader.next(ParquetRecordReader.java:369)
at org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:94)
at org.apache.drill.exec.record.AbstractSingleRecordBatch.next(AbstractSingleRecordBatch.java:42)
at org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.next(ScreenCreator.java:85)
at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:83)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)