[AMQNET-515] Failover is not working

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.7.0
• Component/s: NMS
• Assignee: Unassigned
• Reporter: Maxim Cherednik
• Created: 14/Dec/15 14:27
• Updated: 01/Mar/17 21:17

Description

The client stops reconnecting.

We don't have the exact steps how to reproduce it bu it happens quite often(every second day) in our env.

1. Connection string: <add key="BrokerURI" value="failover:(tcp://192.168.1.2:61617)?transport.timeout=3000&connection.RequestTimeout=15000" />

2. After some time we receive these exceptions on the IConnectionListener
2015-12-15 16:15:52.548 [133] [Main] ERROR Utils.ConnectionListener - Client: Uri: failover:(tcp://192.168.1.2:61617)?transport.timeout=3000&connection.RequestTimeout=15000, Apache.NMS.IllegalStateException: Cannot remove a producer from a session that had not been registered: ID:VM1-57210-635857910536238575-1:5:9613
2015-12-15 16:15:52.549 [133] [Main] ERROR Utils.ConnectionListener - Client: Uri: failover:(tcp://192.168.1.2:61617)?transport.timeout=3000&connection.RequestTimeout=15000, Apache.NMS.IllegalStateException: Cannot remove a producer from a session that had not been registered: ID:VM1-57210-635857910536238575-1:5:9618
2015-12-15 17:03:41.568 [148] [Main] ERROR Utils.ConnectionListener - Client: Uri: failover:(tcp://192.168.1.2:61617)?transport.timeout=3000&connection.RequestTimeout=15000, Apache.NMS.IllegalStateException: Cannot remove a producer from a session that had not been registered: ID:VM1-57210-635857910536238575-1:5:39924
2015-12-16 01:25:33.673 [208] [Main] ERROR Utils.ConnectionListener - Client: Uri: failover:(tcp://192.168.1.2:61617)?transport.timeout=3000&connection.RequestTimeout=15000, Apache.NMS.IllegalStateException: Cannot remove a producer that had not been registered: ID:VM1-57210-635857910536238575-1:5:325989:1
2015-12-16 01:25:39.579 [197] [Main] ERROR Utils.ConnectionListener - Client: Uri: failover:(tcp://192.168.1.2:61617)?transport.timeout=3000&connection.RequestTimeout=15000, Apache.NMS.IllegalStateException: Cannot remove a producer from a session that had not been registered: ID:VM1-57210-635857910536238575-1:5:325987
2015-12-16 01:25:39.982 [197] [Main] ERROR Utils.ConnectionListener - Client: Uri: failover:(tcp://192.168.1.2:61617)?transport.timeout=3000&connection.RequestTimeout=15000, Apache.NMS.IllegalStateException: Cannot remove a producer from a session that had not been registered: ID:VM1-57210-635857910536238575-1:5:325988
2015-12-16 02:29:41.893 [254] [Main] ERROR Utils.ConnectionListener - Client: Uri: failover:(tcp://192.168.1.2:61617)?transport.timeout=3000&connection.RequestTimeout=15000, Apache.NMS.IllegalStateException: Cannot remove a producer that had not been registered: ID:VM1-57210-635857910536238575-1:5:344343:1
2015-12-16 02:29:47.365 [226] [Main] ERROR Utils.ConnectionListener - Client: Uri: failover:(tcp://192.168.1.2:61617)?transport.timeout=3000&connection.RequestTimeout=15000,
Apache.NMS.IllegalStateException: Cannot remove a producer from a session that had not been registered: ID:VM1-57210-635857910536238575-1:5:344346
2015-12-16 02:52:49.925 [251] [Main] ERROR Utils.ConnectionListener - Client: Uri: failover:(tcp://192.168.1.2:61617)?transport.timeout=3000&connection.RequestTimeout=15000,
Apache.NMS.ActiveMQ.IOException: Oneway timed out after 3000 milliseconds.
at Apache.NMS.ActiveMQ.Transport.MutexTransport.GetTransmissionLock(Int32 timeout) in c:\dev\NMS.ActiveMQ\src\main\csharp\Transport\MutexTransport.cs:line 46
at Apache.NMS.ActiveMQ.Transport.MutexTransport.Oneway(Command command) in c:\dev\NMS.ActiveMQ\src\main\csharp\Transport\MutexTransport.cs:line 66
at Apache.NMS.ActiveMQ.Connection.OnKeepAliveCommand(ITransport commandTransport, KeepAliveInfo info) in c:\dev\NMS.ActiveMQ\src\main\csharp\Connection.cs:line 1180

3. After that whenever we are trying to send the message to the broker, we receive this exception:
Apache.NMS.ActiveMQ.RequestTimedOutException: Synchronous Request Timed out after [15000] milliseconds
at Apache.NMS.ActiveMQ.Transport.FutureResponse.get_Response() in c:\dev\NMS.ActiveMQ\src\main\csharp\Transport\FutureResponse.cs:line 78
at Apache.NMS.ActiveMQ.Transport.ResponseCorrelator.Request(Command command, TimeSpan timeout) in c:\dev\NMS.ActiveMQ\src\main\csharp\Transport\ResponseCorrelator.cs:line 92
at Apache.NMS.ActiveMQ.Connection.SyncRequest(Command command, TimeSpan requestTimeout) in c:\dev\NMS.ActiveMQ\src\main\csharp\Connection.cs:line 895
at Apache.NMS.ActiveMQ.Session.DoSend(ActiveMQDestination destination, ActiveMQMessage message, MessageProducer producer, MemoryUsage producerWindow, TimeSpan sendTimeout) in c:\dev\NMS.ActiveMQ\src\main\csharp\Session.cs:line 783
at Apache.NMS.ActiveMQ.MessageProducer.Send(IDestination destination, IMessage message, MsgDeliveryMode deliveryMode, MsgPriority priority, TimeSpan timeToLive, Boolean specifiedTimeToLive) in c:\dev\NMS.ActiveMQ\src\main\csharp\MessageProducer.cs:line 263
at Apache.NMS.ActiveMQ.MessageProducer.Send(IMessage message) in c:\dev\NMS.ActiveMQ\src\main\csharp\MessageProducer.cs:line 163

4. AMQ broker restart doesn't trigger the IConnectionListener methods anymore.