Critical Description
When multiple AMQP clients try to connect to the broker at exactly the same time, the broker can end up in a state where it gets an AMQP parsing error during the connection handshake and then all future AMQP connections fail until the broker is stopped. This was reproduced with C proton clients and QPID JMS clients but it seemed to depend on the speed of the machine where the broker was running and the network speed to ensure that the timing window would be hit. Turning on remote debugging in the ActiveMQ startup script made it happen much more frequently. The QPID JMS clients end up staying hung in the ConnectionEndpoint.open function and the C proton clients return a SASL error. Code analysis in the broker appeared to point to an incorrect use of static for the list data structure in the AMQPProtocolDiscriminator class. I am planning to attach a patch and unit test that demonstrate the behavior as well as some logs. The unit test fails with a "timeout" as all of the threads get hung when attempting to make connection and seems to require the following MAVEN_OPTS to be set to see it fail consistently:
export MAVEN_OPTS="-Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=5005"
When running the broker in console mode, the following would be output for each failed client connection attempt:
Could not decode AMQP frame: hex: 0000002102010000005341d00000001100000002a309414e4f4e594d4f5553a000
Attachments
Attachments
Issue Links
- is duplicated by
-
AMQ-5256 AMQP WARN Transport Connection failed: java.io.IOException
-
- Resolved
-