Apache Drill
  1. Apache Drill
  2. DRILL-473

Memory leak reported in ParquetRecordReader when running a hash aggregation test

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.4.0
    • Component/s: None
    • Labels:
      None

      Description

      This memory leak was detected when working on Drill-443. To reproduce, remove the @Ignore label of testQ8() and testQ8_1() in TestHashAggr.java. See stack below.

      Total 2 allocation(s) of byte size(s): 40000, 40000, at stack location:
      org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.buffer(TopLevelAllocator.java:96)
      org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.buffer(TopLevelAllocator.java:101)
      org.apache.drill.exec.vector.BigIntVector.allocateNew(BigIntVector.java:103)
      org.apache.drill.exec.vector.AllocationHelper.allocate(AllocationHelper.java:29)
      org.apache.drill.exec.store.VectorHolder.allocateNew(VectorHolder.java:90)
      org.apache.drill.exec.store.VectorHolder.reset(VectorHolder.java:73)
      org.apache.drill.exec.store.parquet.ParquetRecordReader.resetBatch(ParquetRecordReader.java:275)
      org.apache.drill.exec.store.parquet.ParquetRecordReader.next(ParquetRecordReader.java:333)
      org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:94)
      org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:107)
      org.apache.drill.exec.record.AbstractSingleRecordBatch.next(AbstractSingleRecordBatch.java:42)
      org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:107)
      org.apache.drill.exec.physical.impl.aggregate.HashAggBatch.next(HashAggBatch.java:105)

        Activity

        Aman Sinha created issue -
        Jake Farrell made changes -
        Field Original Value New Value
        Workflow no-reopen-closed, patch-avail [ 12852928 ] no-reopen-closed, patch-avail, testing [ 12859993 ]
        Jacques Nadeau made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Assignee Aman Sinha [ amansinha100 ]
        Resolution Fixed [ 1 ]
        Jacques Nadeau made changes -
        Fix Version/s 0.4.0 [ 12324963 ]
        Tony Stevenson made changes -
        Workflow no-reopen-closed, patch-avail, testing [ 12859993 ] Drill workflow [ 12934198 ]

          People

          • Assignee:
            Aman Sinha
            Reporter:
            Aman Sinha
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development