Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-3063

TestQueriesOnLargeFile leaks memory with 16M limit

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.0.0
    • Component/s: Execution - Flow
    • Labels:
      None

      Description

      I ran the TestQueriesOnLargeFile unit test with a limited memory environment, limiting direct memory to 16M. At the end of the test, the shutdown hook reports a memory leak.

      Here is the test launch configuration:
      -Xms512m
      -Xmx3g
      -Ddrill.exec.http.enabled=false
      -Ddrill.exec.sys.store.provider.local.write=false
      -Dorg.apache.drill.exec.server.Drillbit.system_options="org.apache.drill.exec.compile.ClassTransformer.scalar_replacement=on"
      -XX:MaxPermSize=256M -XX:MaxDirectMemorySize=3072M
      -XX:+CMSClassUnloadingEnabled -ea
      -Ddrill.exec.memory.top.max=16777216

      Here's what I see at the end:
      at org.apache.drill.exec.server.Drillbit$ShutdownThread.run(Drillbit.java:333)
      Caused by: java.lang.IllegalStateException: Failure while closing accountor. Expected private and shared pools to be set to initial values. However, one or more were not. Stats are
      zone init allocated delta
      private 0 0 0
      shared 16777216 13777216 3000000.
      at org.apache.drill.exec.memory.AtomicRemainder.close(AtomicRemainder.java:200)
      at org.apache.drill.exec.memory.Accountor.close(Accountor.java:386)
      at org.apache.drill.exec.memory.TopLevelAllocator.close(TopLevelAllocator.java:171)
      at org.apache.drill.exec.server.BootStrapContext.close(BootStrapContext.java:75)
      at com.google.common.io.Closeables.close(Closeables.java:77)
      at com.google.common.io.Closeables.closeQuietly(Closeables.java:108)
      at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:292)
      at org.apache.drill.exec.server.Drillbit$ShutdownThread.run(Drillbit.java:330)

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                cwestin Chris Westin
                Reporter:
                cwestin Chris Westin
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: