Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-3065

Memory Leak at ExternalSortBatch

    XMLWordPrintableJSON

    Details

      Description

      This issue would rise if a RuntimeException is injected after mSorter is created but before it does its sort job

      mSorter.setup(context, oContext.getAllocator(), getSelectionVector4(), this.container);
      injector.injectUnchecked(context.getExecutionControls(), "priorSort");
      mSorter.sort(this.container);
      

      Detailed Information:
      1. query:
      select n_name from cp.`tpch/nation.parquet` order by n_name

      2. data:
      cp.`tpch/nation.parquet`

      3. log:
      java.lang.IllegalStateException: Failure while closing accountor. Expected private and shared pools to be set to initial values. However, one or more were not. Stats are
      zone init allocated delta
      private 0 0 0
      shared 3221225472 3195686243 25539229.
      at org.apache.drill.exec.memory.AtomicRemainder.close(AtomicRemainder.java:200)
      at org.apache.drill.exec.memory.Accountor.close(Accountor.java:386)
      at org.apache.drill.exec.memory.TopLevelAllocator.close(TopLevelAllocator.java:171)
      at org.apache.drill.exec.server.BootStrapContext.close(BootStrapContext.java:75)
      at com.google.common.io.Closeables.close(Closeables.java:77)
      at com.google.common.io.Closeables.closeQuietly(Closeables.java:108)
      at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:292)
      at org.apache.drill.BaseTestQuery.closeClient(BaseTestQuery.java:238)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
      at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
      at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
      at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
      at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
      at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
      at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74)
      at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:211)
      at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:67)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)

      Exception in thread "Drillbit-ShutdownHook#0" java.lang.RuntimeException: Caught exception closing Drillbit started from
      org.apache.drill.common.StackTrace.<init>:36
      org.apache.drill.exec.server.Drillbit.run:250
      org.apache.drill.BaseTestQuery.openClient:180
      org.apache.drill.BaseTestQuery.setupDefaultTestCluster:116
      sun.reflect.NativeMethodAccessorImpl.invoke0:-2
      sun.reflect.NativeMethodAccessorImpl.invoke:57
      sun.reflect.DelegatingMethodAccessorImpl.invoke:43
      java.lang.reflect.Method.invoke:606
      org.junit.runners.model.FrameworkMethod$1.runReflectiveCall:47
      org.junit.internal.runners.model.ReflectiveCallable.run:12
      org.junit.runners.model.FrameworkMethod.invokeExplosively:44
      org.junit.internal.runners.statements.RunBefores.evaluate:24
      org.junit.internal.runners.statements.RunAfters.evaluate:27
      org.junit.runners.ParentRunner.run:309
      org.junit.runner.JUnitCore.run:160
      com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs:74
      com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart:211
      com.intellij.rt.execution.junit.JUnitStarter.main:67
      sun.reflect.NativeMethodAccessorImpl.invoke0:-2
      sun.reflect.NativeMethodAccessorImpl.invoke:57
      sun.reflect.DelegatingMethodAccessorImpl.invoke:43
      java.lang.reflect.Method.invoke:606

      at org.apache.drill.exec.server.Drillbit$ShutdownThread.run(Drillbit.java:333)
      Caused by: java.lang.IllegalStateException: Failure while closing accountor. Expected private and shared pools to be set to initial values. However, one or more were not. Stats are
      zone init allocated delta
      private 0 0 0
      shared 3221225472 3195686243 25539229.
      at org.apache.drill.exec.memory.AtomicRemainder.close(AtomicRemainder.java:200)
      at org.apache.drill.exec.memory.Accountor.close(Accountor.java:386)
      at org.apache.drill.exec.memory.TopLevelAllocator.close(TopLevelAllocator.java:171)
      at org.apache.drill.exec.server.BootStrapContext.close(BootStrapContext.java:75)
      at com.google.common.io.Closeables.close(Closeables.java:77)
      at com.google.common.io.Closeables.closeQuietly(Closeables.java:108)
      at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:292)
      at org.apache.drill.exec.server.Drillbit$ShutdownThread.run(Drillbit.java:330)

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                seanhychu Sean Hsuan-Yi Chu
                Reporter:
                seanhychu Sean Hsuan-Yi Chu
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: