Details
Description
If you send a large number of messages inside a transaction, and ActiveMQ runs out of memory, cms::MessageProducer::send() throws an exception (as it should).
The exception text is: No valid response received for command
The problem is ... the last message sent (the one that caused the exception to be thrown) gets added to the queue while the others are rolled back. This is a large problem, as the whole point of transactions is to avoid this sort of thing.
To summarize:
- Start sending messages with the session set to SESSION_TRANSACTED
- Fill ActiveMQ memory
- Receive exception
- The last message sent will be in the queue. All other messages will be rolled back.
Here's what the debug log looks like:
DEBUG Usage - Memory usage change. from: 2, to: 1
DEBUG AMQMessageStore - Transacted message add rollback for: 68445488-66bc-c258-0d30-4a1faede93a5:0:1:109, at: offset = 4527877, file = 6, size = 47444, type = 1
DEBUG Usage - Memory usage change. from: 1, to: 0
DEBUG AMQMessageStore - Transacted message add rollback for: 68445488-66bc-c258-0d30-4a1faede93a5:0:1:110, at: offset = 4575321, file = 6, size = 47439, type = 1
DEBUG TransportConnection - Connection Stopped: /127.0.0.1:44382
DEBUG AMQMessageStore - Journalled message add for: 68445488-66bc-c258-0d30-4a1faede93a5:0:1:111, at: offset = 4622760, file = 6, size = 47442, type = 1
As you can see, it adds a message to the queue after the rollback and after stopping the connection.
Attached is C++ that will demonstrate the problem.
One other thing - the behavior is slightly different with the current 2.3 trunk;
The trunk (2.3?) now blocks when the memory fill occurs. However, if you kill the client, the same problem occurs - the last message winds up in the queue outside of the transaction. And, unfortunately, wrapping the Producer::send() in an alarm and destroying the session on a timeout doesn't solve the problem - still get that last message sent.