Major Description
For some time now there have been various bug reports about ActiveMQ "blocking", "not receiving messages", "running into a deadlock" etc. Since I encoutered such deadlocks now and then, too, I eventually wrote up a JUnit testing scenario for this stuff. I found out that deadlocks can be quite easily reproduced. The symptoms are that the producer thread is sending or committing while the consumer thread is receiving or committing - and none of them can advance. One of the threads is always stuck in a blocking queue.
Here's a sample output of my testing class:
An ActiveMQ deadlock has been discovered. The following threads seem to be involved:
Thread "producer" is inactive since 16 seconds after 358719 status changes. The current status is COMMITTING
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1889)
java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)
org.apache.activemq.transport.FutureResponse.getResult(FutureResponse.java:40)
org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:76)
org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1172)
org.apache.activemq.TransactionContext.commit(TransactionContext.java:259)
org.apache.activemq.ActiveMQSession.commit(ActiveMQSession.java:494)
de.rainer_klute.activemq.ProducerThread.run(ProducerThread.java:162)
Thread "consumer" is inactive since 16 seconds after 1807 status changes. The current status is RECEIVING
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:485)
org.apache.activemq.MessageDispatchChannel.dequeue(MessageDispatchChannel.java:75)
org.apache.activemq.ActiveMQMessageConsumer.dequeue(ActiveMQMessageConsumer.java:404)
org.apache.activemq.ActiveMQMessageConsumer.receive(ActiveMQMessageConsumer.java:452)
org.apache.activemq.ActiveMQMessageConsumer.receive(ActiveMQMessageConsumer.java:504)
de.rainer_klute.activemq.ConsumerThread.run(ConsumerThread.java:183)
The following factors seem to increase the probability of a deadlock:
- small values for memoryUsage
- working transacted in the consumer (not always necessary but "helps")
- many messages in the persistence store (to be achieved via a long delay before the consumer starts to read messages)
