[POOL-85] GenericKeyedObjectPool.getNumIdle() corrupted under high load

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.3
• Fix Version/s: 1.4
• Assignee: Unassigned
• Reporter: Mike Martin
• Created: 08/Oct/06 19:10
• Updated: 05/May/12 22:41
• Resolved: 05/May/12 22:41

Description

GenericKeyedObjectPool can have its getNumIdle() value corrupted (as in
becoming negative) under certain conditions. The conditions appear to be:

o _maxTotal > 0
o Multiple keys are in use
o Objects are borrowed by many threads at a fast rate
o Demand exceeds _maxTotal

When _totalIdle gets corrupted then the "pool exhausted" computations are
compromised, resulting in invalid states such as _totalActive > _maxTotal.

I've determined the problem originates in ObjectTimestampPair; its compareTo()
method is not consistent with equals() (see note in java.lang.Comparable docs
about sorted sets & sorted maps).

When clearOldest() builds a TreeMap using ObjectTimestampPairs as keys the
inconsistency with equals can result in OTPs being mapped to the wrong pool
key if their timestamps are equal. In turn this causes the eventual:

list.remove(pairTimeStamp);

at line 902 to silently attempt removal from the wrong list. The OTP doesn't
get removed but _totalIdle gets decremented.

Here's the patch:

— GenericKeyedObjectPool.java.orig Sun Apr 02 19:59:37 2006
+++ GenericKeyedObjectPool.java Sun Oct 08 13:32:40 2006
@@ -1335,7 +1335,12 @@
}

public int compareTo(ObjectTimestampPair other)

{ - return (int) (this.tstamp - other.tstamp); + int cmp = (int)(this.tstamp - other.tstamp); + + if (cmp == 0) + cmp = System.identityHashCode(this) - System.identityHashCode(other); + + return cmp; }

}

Mike