[DBCP-376] Thread safety issue

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.3
• Fix Version/s: 1.5.1, 2.0
• Assignee: Unassigned
• Reporter: Dave Oxley
• Created: 23/Jan/12 01:04
• Updated: 25/Feb/15 00:27
• Resolved: 25/Feb/15 00:27

Description

Under high load commons-dbcp (or commons-pool) exhibits thread safety issues and begins throwing various exceptions. I don't yet know the cause of the issue but it looks like a connection maybe handed out to multiple threads concurrently. Here's a few examples of the exceptions we are getting:

jvm 1    | Caused by: java.sql.SQLException: Attempted to use PooledConnection after closed() was called.
jvm 1    |      at org.apache.commons.dbcp.cpdsadapter.PooledConnectionImpl.assertOpen(PooledConnectionImpl.java:163)
jvm 1    |      at org.apache.commons.dbcp.cpdsadapter.PooledConnectionImpl.getConnection(PooledConnectionImpl.java:174)
jvm 1    |      at org.apache.commons.dbcp.datasources.InstanceKeyDataSource.getConnection(InstanceKeyDataSource.java:768)
jvm 1    |      at org.apache.commons.dbcp.datasources.InstanceKeyDataSource.getConnection(InstanceKeyDataSource.java:676)
jvm 1    |      at uk.co.webessence.kernel.database.DriverAdapterConnectionPool.acquireConnection(DriverAdapterConnectionPool.java:101)
jvm 1    |      ... 94 more

jvm 1    | Caused by: java.sql.SQLException: PooledConnection was reused, withoutits previous Connection being closed.
jvm 1    |      at org.apache.commons.dbcp.cpdsadapter.PooledConnectionImpl.getConnection(PooledConnectionImpl.java:179)
jvm 1    |      at org.apache.commons.dbcp.datasources.InstanceKeyDataSource.getConnection(InstanceKeyDataSource.java:768)
jvm 1    |      at org.apache.commons.dbcp.datasources.InstanceKeyDataSource.getConnection(InstanceKeyDataSource.java:676)
jvm 1    |      at uk.co.webessence.kernel.database.DriverAdapterConnectionPool.acquireConnection(DriverAdapterConnectionPool.java:101)
jvm 1    |      ... 77 more

jvm 1    | Caused by: java.sql.SQLException: Invalid state, the ResultSet object is closed.
jvm 1    |      at net.sourceforge.jtds.jdbc.JtdsResultSet.checkOpen(JtdsResultSet.java:299)
jvm 1    |      at net.sourceforge.jtds.jdbc.JtdsResultSet.getColumn(JtdsResultSet.java:273)
jvm 1    |      at net.sourceforge.jtds.jdbc.JtdsResultSet.getObject(JtdsResultSet.java:840)
jvm 1    |      at org.apache.commons.dbcp.DelegatingResultSet.getObject(DelegatingResultSet.java:325)
jvm 1    |      at uk.co.webessence.kernel.persistence.Preloader.getDataArray(Preloader.java:428)
jvm 1    |      at uk.co.webessence.kernel.persistence.Preloader.processSingleRow(Preloader.java:175)
jvm 1    |      at uk.co.webessence.kernel.persistence.PersistenceHandler.processRecordReload(PersistenceHandler.java:471)
jvm 1    |      at uk.co.webessence.kernel.persistence.PersistenceHandler$1.doLoad(PersistenceHandler.java:447)
jvm 1    |      ... 71 more

Message:   TDS Protocol error: Invalid packet type 0x4
jvm 1    | Caused by: java.sql.SQLException: TDS Protocol error: Invalid packet type 0x4
jvm 1    |      at net.sourceforge.jtds.jdbc.TdsCore.nextToken(TdsCore.java:2314)
jvm 1    |      at net.sourceforge.jtds.jdbc.TdsCore.getNextRow(TdsCore.java:764)
jvm 1    |      at net.sourceforge.jtds.jdbc.JtdsResultSet.next(JtdsResultSet.java:593)
jvm 1    |      at org.apache.commons.dbcp.DelegatingResultSet.next(DelegatingResultSet.java:207)
jvm 1    |      at uk.co.webessence.kernel.persistence.Preloader.loadData(Preloader.java:142)
jvm 1    |      at uk.co.webessence.kernel.persistence.PersistenceHandler$3.doLoad(PersistenceHandler.java:592)
jvm 1    |      ... 111 more
jvm 1    | Caused by: net.sourceforge.jtds.jdbc.ProtocolException: Invalid packet type 0x4
jvm 1    |      at net.sourceforge.jtds.jdbc.TdsCore.nextToken(TdsCore.java:2301)
jvm 1    |      ... 116 more