Details
Major Description
There seems to be a regression (works fine in 5.2.0) related to the fix for the following:
https://issues.apache.org/activemq/browse/AMQ-2413
I think the intent of the original code was to prevent uncontrolled paging of pending messages. The doPageIn method is called in version 5.3.0 :
- each time a message is acked/committed
- each time the MemoryUsage is decreased
As a result, if you create a queue with, say, 1 000 000 messages in it, then start a consumer, the broker will try to page in all the messages from the store as consuming goes on (with default configuration values, 200 paged in messages for one consumed message minimum - I say minimum because if the MemoryUsage is decreased when you consume a message, then doPageIn is invoked again). Guaranteed OutOfMemoryError as in general consuming messages is slower than paging from store. I guess the same can occur when you have a fast producer and a slow consumer, without producer flow control.
I would say that in any case, the broker should never page in more messages than the maxPageSize attribute, plus the number of dispatched messages. More formally:
int toPageIn = getMaxPageSize() - pagedInPendingDispatch.size();
And just to make sure we never get a negative value:
int toPageIn = Math.max(0, getMaxPageSize() - pagedInPendingDispatch.size());
Note 1: this behavior is also exhibited by the background message expiration logic, that forces message pagination each time it is run. With big queues without consumers, by default you'll get 200 more messages paginated each time the expiration task is run. Eventually leading to broker resource exhaustion if no consumer comes into play in the near future with active producers queuing up messages.
Note 2: this may also be related to https://issues.apache.org/activemq/browse/AMQ-2468
Workarounds:
- setting lazyDispatch to true fixes the problem for message consuming only
- background expiration can be disabled by setting expireMessagesPeriod to -1
Cheers,
Fabrice