Details
Major Description
I have problems using the zero prefetch. The consumer starts prefetching messages after some receives.
Our use case is, that we have a large computation which is split into smaller sub jobs. These jobs are sent via ActiveMQ to some processing nodes on different machines. The duration of jobs differs very much (10s to some minutes). The jobs are sent by decreasing estimated computation time. If one of the consumers which receives a large job prefetches some other jobs, these will be processed later. In the meantime the remaining consumers are idle, and the total computation time is much longer than necessary.
I have modified the existing ZeroPrefetchConsumerTest to test for the problem (I have removed the other test methods).
Two consumers (C1 and C2) are instantiated. These messages are sent:
1,2,3,4,5,6,7,8,9
C1 reads 4 times, receives 1,2,3,4 -> this is correct
now C2 reads, it receives 8 not 5, which is the next message in the queue. The reason is, that C1 prefetched 5, 6, 7, that should not have happened.(sometimes C1 only prefetches 5,6)
The problem can be seen in the JMX Console as well, after a while, the first consumer has more than one dispatched message and the queue has an InflightCount of 3, although there are only two consumers!
The last version that we used was 4.1.1, that worked.