[DIRMINA-1156] Inconsistent worker / idleWorker in OrderedThreadPoolExecutor

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 2.0.23, 2.2.0, 2.1.6
• External issue URL: https://igniterealtime.atlassian.net/browse/OF-2367
• Assignee: Unassigned
• Reporter: Guus der Kinderen
• Created: 12/Jan/22 08:24
• Updated: 06/Feb/22 08:11
• Resolved: 06/Feb/22 07:53

Description

I stumbled upon this while performing analysis of heap dumps of a JVMs that suffered from an issue where no user was able to establish a connection to our server software anymore.
 
Our application uses an ExecutorFilter. The OrderedThreadPoolExecutor of the affected server seems to be in an inconsistent state:

• idleWorkers: 2
• waitingSessions.count: 2293
• workers.map.size: 0

What strikes me as odd is:

• No workers, while there are sessions waiting to be processed
• No workers, but a non-zero idle workers count

Servers that are unaffected by the issue have an idle worker count that is equal to the amount of workers (I assume that the snapshots were taken when that server was not actively processing data).

We are using Apache MINA 2.1.3. I have no indication that this problem is or is not present in other versions.

This issue has also been discussed on the Apache MINA users mailinglist, starting with this message: https://www.mail-archive.com/[e-mail]/msg06887.html