Details
-
Bug
-
Status: Resolved
-
Critical
-
Resolution: Fixed
-
5.13.0
-
None
Description
Potential for deadlock when duplicates read in from store for DLQ destination itself and another producer is trying to send messages from the DLQ destination.
It can result with threads in following state
Found one Java-level deadlock: ============================= "ActiveMQ Transport: tcp:///127.0.0.1:59885@59875": waiting for ownable synchronizer 0x00000007f6188490, (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync), which is held by "ActiveMQ BrokerService[localhost] Task-1" "ActiveMQ BrokerService[localhost] Task-1": waiting for ownable synchronizer 0x00000007f6190cf8, (a java.util.concurrent.locks.ReentrantLock$NonfairSync), which is held by "ActiveMQ Transport: tcp:///127.0.0.1:59885@59875" Java stack information for the threads listed above: =================================================== "ActiveMQ Transport: tcp:///127.0.0.1:59885@59875": at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000007f6188490> (a java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:964) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1282) at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:731) at org.apache.activemq.broker.region.Queue.toString(Queue.java:937) at org.apache.activemq.broker.region.Queue.doMessageSend(Queue.java:830) at org.apache.activemq.broker.region.Queue.send(Queue.java:728) at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:390) at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:455) at org.apache.activemq.broker.jmx.ManagedRegionBroker.send(ManagedRegionBroker.java:297) at org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:96) at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:307) at org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:157) at org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:541) at org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:756) at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:334) at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:188) at org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50) at org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113) at org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:270) at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83) at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:214) at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:196) at java.lang.Thread.run(Thread.java:722) "ActiveMQ BrokerService[localhost] Task-1": at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000007f6190cf8> (a java.util.concurrent.locks.ReentrantLock$NonfairSync) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireInterruptibly(AbstractQueuedSynchronizer.java:894) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1221) at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:340) at org.apache.activemq.broker.region.Queue.doMessageSend(Queue.java:828) at org.apache.activemq.broker.region.Queue.send(Queue.java:728) at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:390) at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:455) at org.apache.activemq.broker.jmx.ManagedRegionBroker.send(ManagedRegionBroker.java:297) at org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:96) at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:307) at org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:157) at org.apache.activemq.util.BrokerSupport.doResend(BrokerSupport.java:68) at org.apache.activemq.util.BrokerSupport.resendNoCopy(BrokerSupport.java:38) at org.apache.activemq.broker.region.RegionBroker.sendToDeadLetterQueue(RegionBroker.java:762) at org.apache.activemq.broker.BrokerFilter.sendToDeadLetterQueue(BrokerFilter.java:318) at org.apache.activemq.broker.BrokerFilter.sendToDeadLetterQueue(BrokerFilter.java:318) at org.apache.activemq.broker.MutableBrokerFilter.sendToDeadLetterQueue(MutableBrokerFilter.java:330) at org.apache.activemq.broker.region.BaseDestination.duplicateFromStore(BaseDestination.java:801) at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.dealWithDuplicates(AbstractStoreCursor.java:134) at org.apache.activemq.broker.region.cursors.QueueStorePrefetch.doFillBatch(QueueStorePrefetch.java:110) at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:385) - locked <0x00000007f6188550> (a org.apache.activemq.broker.region.cursors.QueueStorePrefetch) at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.reset(AbstractStoreCursor.java:142) - locked <0x00000007f6188550> (a org.apache.activemq.broker.region.cursors.QueueStorePrefetch) at org.apache.activemq.broker.region.cursors.StoreQueueCursor.reset(StoreQueueCursor.java:159) - locked <0x00000007f61885a8> (a org.apache.activemq.broker.region.cursors.StoreQueueCursor) at org.apache.activemq.broker.region.Queue.doPageInForDispatch(Queue.java:1906) at org.apache.activemq.broker.region.Queue.pageInMessages(Queue.java:2127) at org.apache.activemq.broker.region.Queue.iterate(Queue.java:1608) - locked <0x00000007f61885f8> (a java.lang.Object) at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133) at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) Found 1 deadlock.