[DRILL-1093] Resource leak: ByteBuf.release() not called

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.4.0
• Fix Version/s: 0.5.0
• Assignee: Jacques Nadeau
• Reporter: Sudheesh Katkam
• Due: 19/Aug/14
• Created: 01/Jul/14 23:16
• Updated: 29/Aug/14 17:55
• Resolved: 29/Aug/14 17:55

Description

0: jdbc:drill:zk=local> SELECT * FROM cp.`employee.json` LIMIT 20;

16:10:39.711 [10a4550e-24cc-44dc-96dc-fe38ad9abb0a:frag:0:0 - Producer Thread] ERROR io.netty.util.ResourceLeakDetector - LEAK: ByteBuf.release() was not called before it's garbage-collected. Enable advanced leak reporting to find out where the leak occurred. To enable advanced leak reporting, specify the JVM option '-Dio.netty.leakDetectionLevel=advanced' or call ResourceLeakDetector.setLevel()

Here's what I get when I set the level to advanced:

16:02:34.268 [c52b9de5-8043-4ba4-a6bc-57df791b9885:frag:0:0 - Producer Thread] ERROR io.netty.util.ResourceLeakDetector - LEAK: ByteBuf.release() was not called before it's garbage-collected.
Recent access records: 2
#2:
io.netty.buffer.AdvancedLeakAwareByteBuf.getBytes(AdvancedLeakAwareByteBuf.java:211)
org.apache.drill.exec.vector.VarCharVector$Mutator.setSafe(VarCharVector.java:494)
org.apache.drill.exec.vector.NullableVarCharVector$Mutator.setSafe(NullableVarCharVector.java:442)
org.apache.drill.exec.vector.complex.impl.NullableVarCharWriterImpl.write(NullableVarCharWriterImpl.java:115)
org.apache.drill.exec.vector.complex.fn.JsonReader.writeData(JsonReader.java:140)
org.apache.drill.exec.vector.complex.fn.JsonReader.write(JsonReader.java:65)
org.apache.drill.exec.vector.complex.fn.JsonReaderWithState.write(JsonReaderWithState.java:52)
org.apache.drill.exec.store.easy.json.JSONRecordReader2.next(JSONRecordReader2.java:90)
org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:138)
org.apache.drill.exec.physical.impl.producer.ProducerConsumerBatch$Producer.run(ProducerConsumerBatch.java:122)
java.lang.Thread.run(Thread.java:745)
#1:
io.netty.buffer.AdvancedLeakAwareByteBuf.setBytes(AdvancedLeakAwareByteBuf.java:319)
org.apache.drill.exec.vector.complex.fn.JsonReader.writeData(JsonReader.java:136)
org.apache.drill.exec.vector.complex.fn.JsonReader.write(JsonReader.java:65)
org.apache.drill.exec.vector.complex.fn.JsonReaderWithState.write(JsonReaderWithState.java:52)
org.apache.drill.exec.store.easy.json.JSONRecordReader2.next(JSONRecordReader2.java:90)
org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:138)
org.apache.drill.exec.physical.impl.producer.ProducerConsumerBatch$Producer.run(ProducerConsumerBatch.java:122)
java.lang.Thread.run(Thread.java:745)
Created at:
io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:55)
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:155)
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:146)
io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:83)
org.apache.drill.exec.vector.complex.fn.JsonReader.writeData(JsonReader.java:135)
org.apache.drill.exec.vector.complex.fn.JsonReader.write(JsonReader.java:65)
org.apache.drill.exec.vector.complex.fn.JsonReaderWithState.write(JsonReaderWithState.java:52)
org.apache.drill.exec.store.easy.json.JSONRecordReader2.next(JSONRecordReader2.java:90)
org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:138)
org.apache.drill.exec.physical.impl.producer.ProducerConsumerBatch$Producer.run(ProducerConsumerBatch.java:122)
java.lang.Thread.run(Thread.java:745)

This happens only on the first query on any json file.