[AMQ-1837] Lost messages after broker recovery

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Duplicate
• Affects Version/s: 5.1.0
• Fix Version/s: 5.3.0
• Component/s: Message Store
• Assignee: Unassigned
• Reporter: Jan Wiemer
• Created: 03/Jul/08 11:21
• Updated: 15/Apr/09 15:56
• Resolved: 15/Apr/09 15:56

Description

We experienced that some messages (send in persistent mode) were lost after the message broker was killed and restarted.
The attached little test simulates the situation:
There are 10 queues, 10 sender and 10 receiver threads each sending / receiving quite frequently to / from one queue for a quite long time.
The receiver for each queue expects the messages exactly in the order the sender has send the messages.
After restarting (killing) the broker the order of the received messages gets disturbed or messages get lost.

• Start the broker as configures by default in the distribution (only switch on JMX support).
• Start the main method of the attached class
• Wait until some messages are received
• kill or shutdown the broker
• restart the broker
• have a look at the output of the test class
  Output

  Exception in ReceiverThread: java.lang.RuntimeException: Received unexpected message! Expected: QUEUE-4: #3197, Received: QUEUE-4: #3267
  

Broker Output

...
INFO  BrokerService                  - Using Persistence Adapter: AMQPersistenceAdapter(D:\sandbox\Experimental\Activemq\apache-activemq-5.2-SNAPSHOT\bin\..\data)
INFO  AMQPersistenceAdapter          - AMQStore starting using directory: D:\sandbox\Experimental\Activemq\apache-activemq-5.2-SNAPSHOT\bin\..\data
INFO  KahaStore                      - Kaha Store using data directory D:\sandbox\Experimental\Activemq\apache-activemq-5.2-SNAPSHOT\bin\..\data\kr-store\state
INFO  AMQPersistenceAdapter          - Active data files: []
WARN  AMQPersistenceAdapter          - The ReferenceStore is not valid - recovering ...
INFO  KahaStore                      - Kaha Store successfully deleted data directory D:\sandbox\Experimental\Activemq\apache-activemq-5.2-SNAPSHOT\bin\..\data\kr-store\data
INFO  AMQPersistenceAdapter          - Journal Recovery Started from: DataManager:(data-)
INFO  KahaStore                      - Kaha Store using data directory D:\sandbox\Experimental\Activemq\apache-activemq-5.2-SNAPSHOT\bin\..\data\kr-store\data
INFO  AMQPersistenceAdapter          - Recovered 198261 operations from redo log in 18.028 seconds.
INFO  AMQPersistenceAdapter          - Finished recovering the ReferenceStore
INFO  BrokerService                  - ActiveMQ 5.2-SNAPSHOT JMS Message Broker (localhost) is starting
...