[POOL-396] Exceptions in validation can cause an object's lifecycle to be stuck in EVICTION state

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.4.2, 2.9.0
• Fix Version/s: 2.11.0
• Assignee: Unassigned
• Reporter: Jeremy Kong
• Created: 14/Jun/21 12:51
• Updated: 22/Jun/21 23:30
• Resolved: 22/Jun/21 23:30

Description

Currently, if validation of an object throws an exception when testing an idle object during eviction, an exception will be thrown from the evict() method. This causes the object to be left in the EVICTION state, but still be present in the idleObjects queue.

Future runs of the eviction thread will run into an infinite loop, because they pick up the afflicted object from the queue, fail to set its state to EVICTION, don't count it as a test, and repeat ad infinitum. This infinite loop means that no further evictions will take place; it also causes increases in garbage collection owing to allocation of the EvictionIterator on each loop iteration.

This test added to TestGenericObjectPool (plus a small change allowing exceptions to be thrown from the validateObject method of SimpleFactory) reproduces the issue.

@Test
@Timeout(value = 60000, unit = TimeUnit.MILLISECONDS)
public void testExceptionInValidationDuringEviction() throws Exception {
    genericObjectPool.setMaxIdle(1);
    genericObjectPool.setMaxTotal(2);
    genericObjectPool.setNumTestsPerEvictionRun(1);
    genericObjectPool.setMinEvictableIdleTime(Duration.ofMillis(0));
    genericObjectPool.setTestWhileIdle(true);

    String active = genericObjectPool.borrowObject();
    genericObjectPool.returnObject(active);

    simpleFactory.setThrowExceptionOnValidate(true);

    try {
        genericObjectPool.evict();
    } catch (Exception e) {
        // OK that a given run fails. But the object should still be evicted.
    }
    genericObjectPool.evict(); // currently fails: causes an infinite loop!
    assertEquals(0, genericObjectPool.getNumActive());
    assertEquals(0, genericObjectPool.getNumIdle());
}

It seems that protection against exceptions thrown by validation code exist elsewhere (e.g. in borrowObject()); similarly, protections against exceptions thrown by user eviction policy code exist in evict(). So it looks like the testWhileIdle() portion of evict() needs to similarly be protected.

Thanks!
Jeremy