Details
-
Bug
-
Status: Resolved
-
Critical
-
Resolution: Fixed
-
None
-
None
Description
This case is related to DRILL-3065.
In ExternalSortBatch, we have an MSorter to do sorting thing. In this class, there are two SelectionVector4, vector4 and aux. If we fail at the time just after either gets new memory, the close() method would fail to clean their allocated memory properly.
To reproduce this problem, inject an exception at the last step of MSortTemplate.setup()
Detailed Information:
1. Query:
select n_name from cp.`tpch/nation.parquet` order by n_name
*. Using this query alone cannot help reproduce the issue. We still need to inject the exception at the right place.
2. Data:
cp.`tpch/nation.parquet`
3. Log:
java.lang.IllegalStateException: Failure while closing accountor. Expected private and shared pools to be set to initial values. However, one or more were not. Stats are
zone init allocated delta
private 0 0 0
shared 3221225472 3195686243 25539229.
at org.apache.drill.exec.memory.AtomicRemainder.close(AtomicRemainder.java:200)
at org.apache.drill.exec.memory.Accountor.close(Accountor.java:386)
at org.apache.drill.exec.memory.TopLevelAllocator.close(TopLevelAllocator.java:175)
at org.apache.drill.exec.server.BootStrapContext.close(BootStrapContext.java:75)
at com.google.common.io.Closeables.close(Closeables.java:77)
at com.google.common.io.Closeables.closeQuietly(Closeables.java:108)
at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:292)
at org.apache.drill.BaseTestQuery.closeClient(BaseTestQuery.java:238)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:211)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:67)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
Exception in thread "Drillbit-ShutdownHook#0" java.lang.RuntimeException: Caught exception closing Drillbit started from
org.apache.drill.common.StackTrace.<init>:36
org.apache.drill.exec.server.Drillbit.run:250
org.apache.drill.BaseTestQuery.openClient:180
org.apache.drill.BaseTestQuery.setupDefaultTestCluster:116
sun.reflect.NativeMethodAccessorImpl.invoke0:-2
sun.reflect.NativeMethodAccessorImpl.invoke:57
sun.reflect.DelegatingMethodAccessorImpl.invoke:43
java.lang.reflect.Method.invoke:606
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall:47
org.junit.internal.runners.model.ReflectiveCallable.run:12
org.junit.runners.model.FrameworkMethod.invokeExplosively:44
org.junit.internal.runners.statements.RunBefores.evaluate:24
org.junit.internal.runners.statements.RunAfters.evaluate:27
org.junit.runners.ParentRunner.run:309
org.junit.runner.JUnitCore.run:160
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs:74
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart:211
com.intellij.rt.execution.junit.JUnitStarter.main:67
sun.reflect.NativeMethodAccessorImpl.invoke0:-2
sun.reflect.NativeMethodAccessorImpl.invoke:57
sun.reflect.DelegatingMethodAccessorImpl.invoke:43
java.lang.reflect.Method.invoke:606
at org.apache.drill.exec.server.Drillbit$ShutdownThread.run(Drillbit.java:333)
Caused by: java.lang.IllegalStateException: Failure while closing accountor. Expected private and shared pools to be set to initial values. However, one or more were not. Stats are
zone init allocated delta
private 0 0 0
shared 3221225472 3195686243 25539229.
at org.apache.drill.exec.memory.AtomicRemainder.close(AtomicRemainder.java:200)
at org.apache.drill.exec.memory.Accountor.close(Accountor.java:386)
at org.apache.drill.exec.memory.TopLevelAllocator.close(TopLevelAllocator.java:175)
at org.apache.drill.exec.server.BootStrapContext.close(BootStrapContext.java:75)
at com.google.common.io.Closeables.close(Closeables.java:77)
at com.google.common.io.Closeables.closeQuietly(Closeables.java:108)
at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:292)
at org.apache.drill.exec.server.Drillbit$ShutdownThread.run(Drillbit.java:330)