Commons Dbcp
  1. Commons Dbcp
  2. DBCP-225

getConnection / borrowObject fails with NullPointerException

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.2.1, 1.2.2
    • Fix Version/s: 1.3
    • Labels:
      None
    • Environment:

      Solaris 10, Oracle 10g RAC

      Description

      We use dbcp PoolingDataSource in Solaris/Oracle 10g RAC environment and our getConnection calls fail sporadically with the following stack trace (1.2.1)

      Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot get a connection, pool exhausted
      at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:103)
      at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:540)
      ... more
      Caused by: java.util.NoSuchElementException: Could not create a validated object, cause: null
      at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:806)
      at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:95)
      ... 24 more

      This is definitely not a "pool exhausted" situation, it is just being reported as pool exhausted. Since NoSuchElementException that you use does not allow cause, only Exception message (null) is being printed. With some debugging I was able to recover the root exception:

      java.lang.NullPointerException
      at org.apache.commons.dbcp.DelegatingConnection.setAutoCommit(DelegatingConnection.java:268)
      at org.apache.commons.dbcp.PoolableConnectionFactory.activateObject(PoolableConnectionFactory.java:368)
      at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:786)
      at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:95)
      at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:540)
      ...

      Looks like it is trying to borrow/validate DelegatingConnection which delegate is null.

      Hoping to resolve the issue we upgraded to 1.2.2 but it did not help. Here is is an exception stack trace from 1.2.2:

      org.apache.commons.dbcp.SQLNestedException: Cannot get a connection, pool error Could not create a validated object, cause: null
      at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:104)
      at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:880)
      ... more
      Caused by: java.util.NoSuchElementException: Could not create a validated object, cause: null
      at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:871)
      at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:96)
      ... 28 more

      We use the following dbcp properties:

      autoCommit="false"
      readOnly="false"
      maxActive="200"
      maxIdle="20"
      minIdle="10"
      minEvictableIdleIime="300000"
      maxWait="200"
      accessToUnderlyingConnectionAllowed="true"
      validationQuery="SELECT 1 FROM DUAL"
      ConnectionCachingEnabled="true"
      FastConnectionFailoverEnabled="true"

      I could not find the existing reported dbcp/object pool bug but I see similar reports on the web, for example
      http://forum.java.sun.com/thread.jspa?threadID=713200&messageID=4124915

      1. DBCP-225.patch
        2 kB
        Dain Sundstrom

        Issue Links

          Activity

          Hide
          Dain Sundstrom added a comment -

          Sending src/java/org/apache/commons/dbcp/PoolableConnectionFactory.java
          Sending src/java/org/apache/commons/dbcp/datasources/CPDSConnectionFactory.java
          Sending src/java/org/apache/commons/dbcp/datasources/KeyedCPDSConnectionFactory.java
          Transmitting file data ...
          Committed revision 558846.

          Show
          Dain Sundstrom added a comment - Sending src/java/org/apache/commons/dbcp/PoolableConnectionFactory.java Sending src/java/org/apache/commons/dbcp/datasources/CPDSConnectionFactory.java Sending src/java/org/apache/commons/dbcp/datasources/KeyedCPDSConnectionFactory.java Transmitting file data ... Committed revision 558846.
          Hide
          Dain Sundstrom added a comment -

          This patch checks for null returned from connection factory and throws an IllegalStateException. We may want to go with a SQLException instead.

          I don't think this will fix Alexei's problem, but at least will cause a fast failure instead of getting a strange "hollow" connection. Alternatively, if we just need to ask Oracle again for a connection, we could put a retry loop in the connection factory.

          Show
          Dain Sundstrom added a comment - This patch checks for null returned from connection factory and throws an IllegalStateException. We may want to go with a SQLException instead. I don't think this will fix Alexei's problem, but at least will cause a fast failure instead of getting a strange "hollow" connection. Alternatively, if we just need to ask Oracle again for a connection, we could put a retry loop in the connection factory.
          Hide
          Alexei Samonov added a comment -

          Yes we were using 1.2 , then we upgraded to 1.3 which did not help. We have to use OracleConnection caching because it is a prerequisite to Oracle's RAC Fast Connection Failure (fcf). We will try to play with Oracle cache parameters such as MaxLimit - it was set to 5 and we will try 200 that would match the max dbcp pool size. I will report test results once I have them.

          Show
          Alexei Samonov added a comment - Yes we were using 1.2 , then we upgraded to 1.3 which did not help. We have to use OracleConnection caching because it is a prerequisite to Oracle's RAC Fast Connection Failure (fcf). We will try to play with Oracle cache parameters such as MaxLimit - it was set to 5 and we will try 200 that would match the max dbcp pool size. I will report test results once I have them.
          Hide
          Phil Steitz added a comment -

          Based on the stack traces, it looks like you are using pool 1.2, correct? From the trace it looks like the NPE is occuring when a newly created connection is being validated. Do you see any connection errors in the database logs? These should throw SQLExceptions, though, which are not being reported. Another thing to try is to turn off Oracle's Connection caching (I assume that is what ConnectionCachingEnabled="true" means).

          Show
          Phil Steitz added a comment - Based on the stack traces, it looks like you are using pool 1.2, correct? From the trace it looks like the NPE is occuring when a newly created connection is being validated. Do you see any connection errors in the database logs? These should throw SQLExceptions, though, which are not being reported. Another thing to try is to turn off Oracle's Connection caching (I assume that is what ConnectionCachingEnabled="true" means).
          Hide
          Anatoliy Salistra added a comment -

          The situation is even more mysterious since the DelegatingConnection in question is PoolableConnection, and no code ever calls setDelegate(null) on PoolableConnection.

          Show
          Anatoliy Salistra added a comment - The situation is even more mysterious since the DelegatingConnection in question is PoolableConnection, and no code ever calls setDelegate(null) on PoolableConnection.

            People

            • Assignee:
              Dain Sundstrom
              Reporter:
              Alexei Samonov
            • Votes:
              1 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development