Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
5.15.0
-
None
Description
With a blocked socket write, the failover (use for masterslave) transport holds the reconnect mutex. this blocks normal stop pending the completion of the write. If the write blocks and there is no timeout exception, the transport stays up in error even when stop is called on the network bridge which can block broker shutdown.
Threads:
"ActiveMQ BrokerService[amq10] Task-147" - Thread t@45016 java.lang.Thread.State: RUNNABLE at java.net.SocketOutputStream.socketWrite0(Native Method) at java.net.SocketOutputStream.__AW_socketWrite(SocketOutputStream.java:111) at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java) at java.net.SocketOutputStream.write(SocketOutputStream.java:155) at sun.security.ssl.OutputRecord.writeBuffer(OutputRecord.java:431) at sun.security.ssl.OutputRecord.write(OutputRecord.java:417) at sun.security.ssl.SSLSocketImpl.writeRecordInternal(SSLSocketImpl.java:886) at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:857) at sun.security.ssl.AppOutputStream.__AW_write(AppOutputStream.java:123) - locked <5ce0bde7> (a sun.security.ssl.AppOutputStream) at sun.security.ssl.AppOutputStream.write(AppOutputStream.java) at org.apache.activemq.transport.tcp.TcpBufferedOutputStream.flush(TcpBufferedOutputStream.java:115) at java.io.DataOutputStream.flush(DataOutputStream.java:123) at org.apache.activemq.transport.tcp.TcpTransport.oneway(TcpTransport.java:176) at org.apache.activemq.transport.AbstractInactivityMonitor.doOnewaySend(AbstractInactivityMonitor.java:335) at org.apache.activemq.transport.AbstractInactivityMonitor.oneway(AbstractInactivityMonitor.java:317) at org.apache.activemq.transport.TransportFilter.oneway(TransportFilter.java:85) at org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:116) at org.apache.activemq.transport.failover.FailoverTransport.__AW_oneway(FailoverTransport.java:667) - locked <47e37945> (a java.lang.Object) at org.apache.activemq.transport.failover.FailoverTransport.oneway(FailoverTransport.java) at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:68) at org.apache.activemq.transport.ResponseCorrelator.asyncRequest(ResponseCorrelator.java:81) at org.apache.activemq.network.DemandForwardingBridgeSupport.serviceLocalCommand(DemandForwardingBridgeSupport.java:1008) at org.apache.activemq.network.DemandForwardingBridgeSupport$2.onCommand(DemandForwardingBridgeSupport.java:207) at org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:116) at org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50) at org.apache.activemq.transport.vm.VMTransport.doDispatch(VMTransport.java:162) at org.apache.activemq.transport.vm.VMTransport.dispatch(VMTransport.java:154) - locked <467f411f> (a java.util.concurrent.atomic.AtomicBoolean) at org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:131) at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:68) at org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60) at org.apache.activemq.broker.TransportConnection.dispatch(TransportConnection.java:1457) at org.apache.activemq.broker.TransportConnection.processDispatch(TransportConnection.java:971) at org.apache.activemq.broker.TransportConnection.iterate(TransportConnection.java:1017) at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133) at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
"IOExceptionHandler: stopping BrokerService[amq10]" - Thread t@201006 java.lang.Thread.State: BLOCKED at org.apache.activemq.transport.failover.FailoverTransport.__AW_stop(FailoverTransport.java:380) - waiting to lock <47e37945> (a java.lang.Object) owned by "ActiveMQ BrokerService[amq10] Task-147" t@45016 at org.apache.activemq.transport.failover.FailoverTransport.stop(FailoverTransport.java) at org.apache.activemq.transport.TransportFilter.stop(TransportFilter.java:65) at org.apache.activemq.transport.TransportFilter.stop(TransportFilter.java:65) at org.apache.activemq.transport.ResponseCorrelator.stop(ResponseCorrelator.java:132) at org.apache.activemq.util.ServiceStopper.stop(ServiceStopper.java:41) at org.apache.activemq.network.DemandForwardingBridgeSupport.__AW_stop(DemandForwardingBridgeSupport.java:307) at org.apache.activemq.network.DemandForwardingBridgeSupport.stop(DemandForwardingBridgeSupport.java) at org.apache.activemq.network.DiscoveryNetworkConnector.handleStop(DiscoveryNetworkConnector.java:223) at org.apache.activemq.network.NetworkConnector$1.doStop(NetworkConnector.java:65) at org.apache.activemq.util.ServiceSupport.__AW_stop(ServiceSupport.java:71) at org.apache.activemq.util.ServiceSupport.stop(ServiceSupport.java) at org.apache.activemq.network.NetworkConnector.stop(NetworkConnector.java:165) at org.apache.activemq.util.ServiceStopper.stop(ServiceStopper.java:41) at org.apache.activemq.broker.BrokerService.__AW_stopAllConnectors(BrokerService.java:2191) at org.apache.activemq.broker.BrokerService.stopAllConnectors(BrokerService.java) at org.apache.activemq.broker.BrokerService.__AW_stop(BrokerService.java:805) at org.apache.activemq.broker.BrokerService.stop(BrokerService.java) at org.apache.activemq.xbean.XBeanBrokerService.stop(XBeanBrokerService.java:122) at org.apache.activemq.util.DefaultIOExceptionHandler$2.run(DefaultIOExceptionHandler.java:181)