[AMQ-3885] ActiveMQ java client doesn't scale to thousands of queues

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 5.6.0
• Fix Version/s: 5.7.0
• Component/s: Connector
• Labels: ActiveMQConnection TaskRunnerFactory client java

Environment

core i7 laptop running win7 64bit with 8gb of ram

• Assignee: Dejan Bosanac
• Reporter: Matan Zruya
• Created: 17/Jun/12 12:59
• Updated: 20/Jul/12 14:53
• Resolved: 20/Jul/12 14:47

Description

The ActiveMQ broker scales to tens of thousands of queues easily when using -Dorg.apache.activemq.UseDedicatedTaskRunner=false (false by default).

A problem actually arises in the java client side, when a client is listening to X queues using 1 connection and Y sessions per queue, using a JMS message listener, X * Y threads will be created, when X * Y is not bounded,

This is because each ActiveMQConnection object holds a TaskRunnerFactory which in turn has a ThreadPoolExecutor, the max pool size of the executor is defined to be Integer.MAX_VALUE, with the combination of a SynchronousQueue it creates as many threads as it pleases.

the behavior of the executor is dependent on min pool size, core pool size, queue impl and rejection policy

a few options to handle this :
1. the default rejection policy throws an exception which is not handled,
ThreadPoolExecutor.CallerRunsPolicy sounds better then the default

2. Change the queue impl to LinkedBlockingQueue and change the core pool size to a bounded value

3. Provide the means to supply an external executor to be shared in the app