ActiveMQ
  1. ActiveMQ
  2. AMQ-1837

Lost messages after broker recovery

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Critical Critical
    • Resolution: Duplicate
    • Affects Version/s: 5.1.0
    • Fix Version/s: 5.3.0
    • Component/s: Message Store
    • Labels:
      None

      Description

      We experienced that some messages (send in persistent mode) were lost after the message broker was killed and restarted.
      The attached little test simulates the situation:
      There are 10 queues, 10 sender and 10 receiver threads each sending / receiving quite frequently to / from one queue for a quite long time.
      The receiver for each queue expects the messages exactly in the order the sender has send the messages.
      After restarting (killing) the broker the order of the received messages gets disturbed or messages get lost.

      • Start the broker as configures by default in the distribution (only switch on JMX support).
      • Start the main method of the attached class
      • Wait until some messages are received
      • kill or shutdown the broker
      • restart the broker
      • have a look at the output of the test class
        Output
        Exception in ReceiverThread: java.lang.RuntimeException: Received unexpected message! Expected: QUEUE-4: #3197, Received: QUEUE-4: #3267
        
      Broker Output
      ...
      INFO  BrokerService                  - Using Persistence Adapter: AMQPersistenceAdapter(D:\sandbox\Experimental\Activemq\apache-activemq-5.2-SNAPSHOT\bin\..\data)
      INFO  AMQPersistenceAdapter          - AMQStore starting using directory: D:\sandbox\Experimental\Activemq\apache-activemq-5.2-SNAPSHOT\bin\..\data
      INFO  KahaStore                      - Kaha Store using data directory D:\sandbox\Experimental\Activemq\apache-activemq-5.2-SNAPSHOT\bin\..\data\kr-store\state
      INFO  AMQPersistenceAdapter          - Active data files: []
      WARN  AMQPersistenceAdapter          - The ReferenceStore is not valid - recovering ...
      INFO  KahaStore                      - Kaha Store successfully deleted data directory D:\sandbox\Experimental\Activemq\apache-activemq-5.2-SNAPSHOT\bin\..\data\kr-store\data
      INFO  AMQPersistenceAdapter          - Journal Recovery Started from: DataManager:(data-)
      INFO  KahaStore                      - Kaha Store using data directory D:\sandbox\Experimental\Activemq\apache-activemq-5.2-SNAPSHOT\bin\..\data\kr-store\data
      INFO  AMQPersistenceAdapter          - Recovered 198261 operations from redo log in 18.028 seconds.
      INFO  AMQPersistenceAdapter          - Finished recovering the ReferenceStore
      INFO  BrokerService                  - ActiveMQ 5.2-SNAPSHOT JMS Message Broker (localhost) is starting
      ...
      
      1. ASF.LICENSE.NOT.GRANTED--src.zip
        2 kB
        Jan Wiemer
      2. MultithreadedQueueTest2.zip
        4 kB
        Helmut Janknecht

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Unassigned
            Reporter:
            Jan Wiemer
          • Votes:
            2 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development