[POOL-213] _numActive can go negative

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.5.4
• Fix Version/s: 2.0
• Assignee: Unassigned
• Reporter: Mark Mindenhall
• Created: 28/Mar/12 18:48
• Updated: 31/Dec/14 16:31
• Resolved: 10/Jan/13 15:34

Description

I'm working on a project that uses Hector (Cassandra client). Hector uses commons-pool (we're using 1.5.4) to pool connections to hosts within a Cassandra cluster. Hector provides a JMX MBean that exposes a "NumActive" property, which is the cumulative call to retrieve numActive from all of the individual connection pools. When querying this property via JMS on our production servers, we often see negative values. For example, on a server that has three connection pools, the "NumActive" property reported was -3899.

I know this issue has been reported before (POOL-29), and was supposedly fixed. The fix discussed there was to merely check the value of _numActive to prevent it from going negative. However, that does not fix the real problem here, which is that it is possible to decrement _numActive more than once for each activated object.

For example, from a quick look at the code (GenericObjectPool.java, v1.5.4), it would be possible to do the following:

1) Create a pool with 10 objects.
2) Borrow all 10 objects from the pool.
3) Call getNumActive (returns 10).
4) Call invalidateObject for ONE of the objects 11 times.
5) Call getNumActive (returns -1).

The invalidateObject method calls the _factory to destroy the object, and subsequent calls to destroy the same object may or may not result in an exception. Regardless, _numActive is decremented within a finally block, and therefore would always be decremented even if the object had already been invalidated and destroyed.

I'd like to suggest using a HashSet instead of a counter to keep track of active objects. If borrowing an object added it to a HashSet, and returning or invaliding the object removed it from the HashSet (subsequent removes would be no-ops), the example given above would not result in an incorrect value when getNumActive is called (it would just return the current size of the HashSet).

Note that although unrelated to this bug, it might also be wise to use a HashSet instead of the int counter _numInternalProcessing.

Issue Links

• is depended upon by: DBCP-390 calling invalidObject(badConnection) damage the counting of the objects in the pool.