Details
-
Bug
-
Status: Closed
-
Critical
-
Resolution: Fixed
-
5.14.1
-
Documentation
-
Patch Available
Description
The chapter "Pooled Consumers and Prefetch" on http://activemq.apache.org/what-is-the-prefetch-limit-for.html is partly incorrect.
I had a few customers getting uncertain about this section that I really think we need to correct this.
While the first general part is correct:
Consuming messages from a pool of consumers an be problematic due to prefetch. Unconsumed prefetched messages are only released when a consumer is closed, but with a pooled consumer the close is deferred (for reuse) till the consumer pool closes. This leaves prefetched messages unconsumed till the consumer is reused. This feature can be desirable from a performance perspective. However, it can lead to out-of-order message delivery when there is more than one consumer in the pool. For this reason, the org.apache.activemq.pool.PooledConnectionFactory does not pool consumers.
the latter part is wrong:
The problem is visible with the Spring DMLC when the cache level is set to CACHE_CONSUMER and there are multiple concurrent consumers. One solution to this problem is to use a prefetch of 0 for a pooled consumer, in this way, it will poll for messages on each call to receive(timeout). Another option is to enable the AbortSlowAckConsumerStrategy on the broker to disconnect consumers that have not acknowledged a Message after some configurable time period.
Springs DMLC does not pool consumers in the sense that it uses its own pool of JMS consumers for which it grabs an instance when consuming the next message. If it did work that way, then it would fall under the above category and then it would be a problem.
However Spring DMLC caches the consumer, i.e. it re-uses the same JMS consumer object to receive all messages for the life time of the DMLC instance. So it behaves pretty much like properly hand written JMS code, where you create the JMS connection, session, consumer and then use this consumer instance to receive all your messages.
If you run Spring DMLC with multiple consumer threads, then each consumer thread creates its own JMS consumer instances and uses that to process messages. These JMS consumer instances are not mixed between threads.
So there is no problem with prefetching messages to these consumers, since they are never placed into a pool of some sort.
On the contrary Springs CachingConnectionFactory allows to pool consumers (although off by default) and that kind of pooling of consumers would cause the problems outlined in the general section.