[AMQ-6688] org.apache.activemq.broker.region.Queue.doMessageSend() future not completing

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 5.14.0
• Fix Version/s: 5.15.0
• Component/s: Broker, KahaDB, Message Store
• Assignee: Gary Tully
• Reporter: Gary Tully
• Created: 29/May/17 16:10
• Updated: 29/May/17 16:16
• Resolved: 29/May/17 16:16

Description

A case of a send thread blocked on the send future.get()...

ActiveMQ VMTransport: vm://XXXXXX" #375 daemon prio=5 os_prio=0 tid=0x00007f9bdc2f0800 nid=0x4e2 waiting on condition [0x00007f9bc7e9e000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x00000000d8e65ef8> (a org.apache.activemq.store.kahadb.KahaDBStore$StoreQueueTask$InnerFutureTask)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
	at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)
	at java.util.concurrent.FutureTask.get(FutureTask.java:191)
	at org.apache.activemq.broker.region.Queue.doMessageSend(Queue.java:853)
	at org.apache.activemq.broker.region.Queue.send(Queue.java:727)
	at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:419)
	at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:468)
	at org.apache.activemq.broker.jmx.ManagedRegionBroker.send(ManagedRegionBroker.java:296)
	at org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:152)
	at org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:96)
	at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:293)
	at org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:157)
	at org.apache.activemq.broker.util.LoggingBrokerPlugin.send(LoggingBrokerPlugin.java:275)
	at org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:152)
	at org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:157)
	at org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:571)
	at org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:768)
	at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:326)
	at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:190)
	at org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:116)
	at org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)
	at org.apache.activemq.transport.vm.VMTransport.iterate(VMTransport.java:271)
	at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133)
	at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

It seems that the org.apache.activemq.store.kahadb.KahaDBStore.StoreQueueTask#run() which should invoke complete on the future has been invoked but complete is not triggered. This can happen in the event that the add results in a Throwable!