number of connections created has crossed more than maxActive

• Key: POOL-107
• Type: Bug
• Status: Resolved
• Resolution: Fixed
• Priority: Major
• Assignee: Unassigned
• Reporter: Bhaskar NA
• Affects Version/s: 1.3
• Fix Version/s: 1.5

Environment

OS:Red Hat Enterprise Linux AS release 4 (Nahant Update 2)
uname:Linux 2.6.9-22.ELsmp #1 SMP Mon Sep 19 18:32:14 EDT 2005 i686
vm_info: Java HotSpot(TM) Server VM (1.4.2_13-b06) for
linux-x86, built
on Oct 18 2006 09:55:11 by unknown with unknown compiler

Description

I found that when the minIdle is configured then during loads, common pool creates more number of connections it will be greater than maxActive.

When I had following settings:

maxActive = 50
maxIdle = 30
minIdle=30
factory=org.apache.commons.dbcp.BasicDataSourceFactory
maxWait=10000
timeBetweenEvictionRunsMillis=900000
numTestsPerEvictionRun=50
minEvictableIdleTimeMillis=1800000
testWhileIdle=true
testOnBorrow = true
validationQuery='select 1 from dual'

Number of connections in the pool went upto 121. (found thru netstat)

When I changed minIdle to 15 and did my load test connecitons in the pool went upto 66

Looks like maxActive is getting bypassed with minIdle.

When I changed minIdle to 10 and maxActive to 30 and maxIdle to 30 then number of connections went upto 40.

Can someone please throw light on what is going on here? I am using DBCP 1.2.1 and common pool 1.3.

Bhaskar

Comments

[Bhaskar NA]

When I changed minIdle to 0 and maxActive to 30 and maxIdle to 30 then number of connections went upto 29.

In all the above tests, i had 40 concurrent threads that intern caused application code to get a connection from pool.

[Bhaskar NA]

Today i tried again with common DBCP 1.2.2 but found the same issue.
I had
maxIdle=30
minIdle=30
maxActive=30

after doing the loadtest, i found in netstat that there were 120 connections established.

I don't see an issue of crossing the maxActive if i set minIdle to 0.

[Phil Steitz]

The pool code to ensure minIdle should not cause this to happen.
Can you provide some more information on your setup and load test? In particular,
How is the DBCP datasource being instantiated? Is DBCP running in a web container?
What is the database? Is it remote from the host running the pool? Are all of the TCP connections ESTABLISHED? Are you sure they are all associated with the client connection pool?
Are client threads timing out waiting for connections during the test (i.e.., do you see org.apache.commons.dbcp.SQLNestedException: Cannot get a connection, pool error Timeout waiting for idle object)?
What does the DBCP client do with connections obtained during the test?
Is the load constant, with threads just repeating the same requests with no delays in between requests?
What is the duration of the test?

[Bhaskar NA]

How is the DBCP datasource being instantiated? Is DBCP running in a web container?

In tomcat's server.xml, in ResourceParams, factory param is configured to use org.apache.commons.dbcp.BasicDataSourceFactory.
In the code, we do the jndi lookup for the above ds and call ds.getConnection().

What is the database? Is it remote from the host running the pool?
oracle is the database and it is in a remote machine

Are all of the TCP connections ESTABLISHED?
yes

Are you sure they are all associated with the client connection pool?
yes.

Are client threads timing out waiting for connections during the test (i.e.., do you see org.apache.commons.dbcp.SQLNestedException: Cannot get a connection, pool error Timeout waiting for idle object)?

Yes, if i have too many concurrent threads then i use to hit this issue.

What does the DBCP client do with connections obtained during the test?
query/update's
and then giveback connection to pool

Is the load constant, with threads just repeating the same requests with no delays in between requests?
load was constant.
they use to repeat same work

What is the duration of the test?
30 min.

I can't reproduce or verify this as a pool or dbcp bug. The only way I can understand this happening is if connections or associated sockets are failing to close. I have not been able to reproduce this behavior with Oracle, MySql or postgres, using same or similar settings to above.
Leaving open but changing fix version to 2.0.

[Mark Thomas]

I can re-create this, but only by forcing a particular execution sequence with a debugger. The issue lies within the for loop in ensureMinIdle(). Consider the case where _numIdle=0, _minIdle>0 and _numActive+1 == _maxActive. If a client thread enters borrowObject() at the point where the evictor thread is about to call addObject() both threads will create objects leaving _numIdle==1 & _numActive==_maxActive. The next client call to borrowObject() will result in _numActive==_maxActive+1

DBCP-44 contains a patch that addresses this for GenericObjectPool. The other pools need a similar patch. Since it is pool that needs to be patched, I'll use this issue to track progress rather than DBCP-44.

[Mark Thomas]

This has been fixed for both GOP and GKOP based on the patch attached DBCP-44.

[Krystian Szczesny]

Hi
We were using commons-pool 1.4 in our application and we came across this issue.
Unfortunately it is critical for us :/

When I have a setting of:
max active: 10
max:idle 5
min idle:2

When running test with 20 threads I get:
before test is ran:
num active: 0
num idle: 2

during test:
num active: 20
num idle: 0

after the test is done:
num active: 0
num idle: 5

I've updated commons-pool to 1.5.2 and unfortunately same thing happens, I cannot control maximum of active connections!
I've tried changing the min idle and max idle to many different settings but was never able to limit the number of active connections!

[Mark Thomas]

Please provide some sample code that demonstrates the issue.

[Krystian Szczesny]

I'm sorry Mark,

I came back here to remove my comment as after building the app from scratch I have found that by default it used
WhenExhaustedAction = GROW and not BLOCK!

Sorry about that, tiredness.
It works fine now!

[Mark Thomas]

No problem. Good to hear 1.5.2 is working for you.