[DRILL-1735] JDBC driver's close() leaks resources in local mode

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 0.8.0
• Component/s: Client - JDBC
• Assignee: Daniel Barclay
• Reporter: Daniel Barclay
• Created: 17/Nov/14 18:28
• Updated: 25/May/18 07:16
• Resolved: 20/Mar/15 05:23

Description

When the Drill JDBC driver is used in local mode (as DriverManager.getConnection("jdbc:drill:zk=local", ... ), the close() method doesn't free resources: Calling getConnection() and then close() and then repeating that pair of calls enough times results in an exception, the root cause exeception of which is an IOException saying "Too many open files":

java.sql.SQLException: Failure while attempting to start Drillbit in embedded mode.
at org.apache.drill.jdbc.DrillConnectionImpl.<init>(DrillConnectionImpl.java:81)
at org.apache.drill.jdbc.DrillJdbc41Factory$DrillJdbc41Connection.<init>(DrillJdbc41Factory.java:88)
at org.apache.drill.jdbc.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:57)
at org.apache.drill.jdbc.DrillJdbc41Factory.newDrillConnection(DrillJdbc41Factory.java:1)
at org.apache.drill.jdbc.DrillFactory.newConnection(DrillFactory.java:51)
at net.hydromatic.avatica.UnregisteredDriver.connect(UnregisteredDriver.java:126)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:187)
at temp.Bug1735CloseDoesntCloseTest.testBug(Bug1735CloseDoesntCloseTest.java:26)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.lang.reflect.Method.invoke(Method.java:606)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
Caused by: java.lang.IllegalStateException: failed to create a child event loop
at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:68)
at io.netty.channel.MultithreadEventLoopGroup.<init>(MultithreadEventLoopGroup.java:49)
at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:61)
at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:52)
at org.apache.drill.exec.rpc.TransportCheck.createEventLoopGroup(TransportCheck.java:75)
at org.apache.drill.exec.server.BootStrapContext.<init>(BootStrapContext.java:45)
at org.apache.drill.exec.server.Drillbit.<init>(Drillbit.java:95)
at org.apache.drill.jdbc.DrillConnectionImpl.<init>(DrillConnectionImpl.java:78)
... 18 more
Caused by: io.netty.channel.ChannelException: failed to open a new selector
at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:128)
at io.netty.channel.nio.NioEventLoop.<init>(NioEventLoop.java:120)
at io.netty.channel.nio.NioEventLoopGroup.newChild(NioEventLoopGroup.java:87)
at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:64)
... 25 more
Caused by: java.io.IOException: Too many open files
at sun.nio.ch.IOUtil.makePipe(Native Method)
at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:126)
... 28 more

Issue Links

• contains: DRILL-2472 JDBC : ResultSet.wasNull should give proper error message when a user tries to call it before calling getXXX method
• contains: DRILL-2319 JDBC DrillStatementRegistry unimplemented; Statements leak, throw errors
• depends upon: DRILL-2245 Improve Drillbit resiliency
• incorporates: DRILL-2319 JDBC DrillStatementRegistry unimplemented; Statements leak, throw errors
• is related to: DRILL-6207 Too many open files in distributed mode
• is related to: DRILL-2231 CTAS (Parquet) followed by a select query fails via JDBC
• relates to: DRILL-2447 Calling getObject on a closed ResultSet object should throw a SQLException
• relates to: DRILL-2444 JDBC creates non-daemon threads that outlive user's code