[DRILL-2510] Unclosed allocators detected in Java shutdown hook

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.7.0
• Fix Version/s: 0.8.0
• Component/s: Execution - Flow
• Assignee: Jacques Nadeau
• Reporter: Chris Westin
• Created: 20/Mar/15 18:36
• Updated: 21/Mar/15 01:14
• Resolved: 21/Mar/15 01:14

Description

We're seeing errors in logs that look like this:

Exception in thread "Drillbit-ShutdownHook#1" 
java.lang.IllegalStateException: Failure while trying to close allocator: Child level allocators not closed. Stack trace:
                java.lang.Thread.getStackTrace(Thread.java:1589)
                org.apache.drill.exec.memory.TopLevelAllocator.getChildAllocator(TopLevelAllocator.java:126)
                org.apache.drill.exec.ops.FragmentContext.<init>(FragmentContext.java:111)
                org.apache.drill.exec.work.foreman.Foreman.setupRootFragment(Foreman.java:763)
                org.apache.drill.exec.work.foreman.Foreman.runPhysicalPlan(Foreman.java:349)
                org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:737)
                org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:202)
                org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
                java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
                java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
                java.lang.Thread.run(Thread.java:745)

        at org.apache.drill.exec.memory.TopLevelAllocator.close(TopLevelAllocator.java:159)
        at org.apache.drill.exec.server.BootStrapContext.close(BootStrapContext.java:75)
        at com.google.common.io.Closeables.close(Closeables.java:77)
        at com.google.common.io.Closeables.closeQuietly(Closeables.java:108)
        at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:267)
        at org.apache.drill.exec.server.Drillbit$ShutdownThread.run(Drillbit.java:285)

This might be a result of sharing the jvm across test classes during mvn install runs, but it might also be a new problem, because some examples can be seen in individual tests run from IDEs.

Investigate.

Issue Links

• relates to: DRILL-2515 Add allocator state verification at the end of test suites