Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-8483

SpilledRecordBatch memory leak when the program threw an exception during the process of building a hash table

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.21.1
    • 1.21.2
    • Server
    • None

    Description

      During the process of reading data from disk to building hash tables in memory, if an exception is thrown, it will result in a memory  SpilledRecordBatch leak

      exception log as following

      Caused by: org.apache.drill.exec.exception.OutOfMemoryException: Unable to allocate buffer of size 8192 due to memory limit (41943040). Current allocation: 3684352
              at org.apache.drill.exec.memory.BaseAllocator.buffer(BaseAllocator.java:241)
              at org.apache.drill.exec.memory.BaseAllocator.buffer(BaseAllocator.java:216)
              at org.apache.drill.exec.vector.VarCharVector.allocateNew(VarCharVector.java:411)
              at org.apache.drill.exec.vector.NullableVarCharVector.allocateNew(NullableVarCharVector.java:270)
              at org.apache.drill.exec.physical.impl.common.HashPartition.allocateNewVectorContainer(HashPartition.java:215)
              at org.apache.drill.exec.physical.impl.common.HashPartition.allocateNewCurrentBatchAndHV(HashPartition.java:238)
              at org.apache.drill.exec.physical.impl.common.HashPartition.<init>(HashPartition.java:165)

       

      Attachments

        Activity

          People

            Unassigned Unassigned
            shihuafeng shihuafeng
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: