[DRILL-1322] Memory leak when first batch is filtered out completely

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.5.0
• Assignee: DrillCommitter
• Reporter: Mehant Baid
• Created: 22/Aug/14 01:28
• Updated: 29/Aug/14 17:55
• Resolved: 29/Aug/14 17:55

Description

Simple repro:

Create a directory with two files:

a.json

{"col" : 1}

b.json

{"col" : 2}

select col as col1 from dfs.`/tmp/data2` where col = 2

Removing the column aliasing seems to get rid of the problem.

Following is the allocation stack:
org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.buffer(TopLevelAllocator.java:159) org.apache.drill.exec.vector.BigIntVector.allocateNewSafe(BigIntVector.java:135) org.apache.drill.exec.vector.BigIntVector.allocateNew(BigIntVector.java:121) org.apache.drill.exec.vector.NullableBigIntVector.allocateNew(NullableBigIntVector.java:152) org.apache.drill.exec.test.generated.CopierGen1.copyRecords(CopierTemplate2.java:45) org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.doWork(RemovingRecordBatch.java:103) org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:78) org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext(RemovingRecordBatch.java:97) org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:95) org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:116) org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:75) org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:65) org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:45) org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:120) org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:95) org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:116) org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:59) org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:98) org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:49) org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:105) org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:250) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:744)