Uploaded image for project: 'ActiveMQ Classic'
  1. ActiveMQ Classic
  2. AMQ-3405

DLQ messages moved/copied into original queue disappear after failing processing for a second time

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 5.4.2
    • 5.9.0
    • Broker
    • None
    • Windows 7, Java 64bit 1.6 update 20, Client and server 5.4.2

    Description

      1. Message in queue A fails enough times to be moved to the DLQ.
      2. We move/copy the message back to queue A to attempt reprocessing. Invoking the retry operation from JMX can also be used.
      3. The consumer once again fails to consume the message greater than the max number of times.
      4. The message is not put back into the DLQ. If it was moved rather than copied, it means we have a message that was never successfully consumed, but has nevertheless disappeared.

      This is all with the default ActiveMQ settings, though I tried adjusted the DLQ configuration to support both expired and non-persistent messages.

      I found this posting which seems to indicate the same issue I am seeing: http://www.mail-archive.com/users@activemq.apache.org/msg09683.html. One difference that I am seeing is that restarting ActiveMQ does not cause the message to show up in the DLQ again. Once it is lost, it is lost forever.

      To replicate this, I believe you can use a consumer that just fails on every invocation.

      One interesting thing is that if I put a message into the DLQ directly, then move it to a queue with a consumer, it will end up in the DLQ again as you would expect. If I then move the DLQ message back to the original queue, it will then show the bevahiour outlined above.

      Attachments

        Issue Links

          Activity

            It would be really helpful if you could take a stab at creating a JUnit test to reproduce the issue, this will also ensure that if it gets fixed it won't break again.

            tabish Timothy A. Bish added a comment - It would be really helpful if you could take a stab at creating a JUnit test to reproduce the issue, this will also ensure that if it gets fixed it won't break again.

            I can confirm that this issue also affects version 5.5.0. It also occurs for any message created by the admin console, they just never go to the DLQ at all.

            Does anyone know of a workaround?

            sbrandwoo Stephen Brandwood added a comment - I can confirm that this issue also affects version 5.5.0. It also occurs for any message created by the admin console, they just never go to the DLQ at all. Does anyone know of a workaround?

            The reason the Message do not reappear in the DLQ is due to Message Audit on the DLQ filtering what would be duplicate messages. I've created a test case and included it in trunk to show that if Audit is disabled on the DLQ then the Messages are moved back in. The Move operation places the message onto the destination without modifying its Message Id so any Audited resource will filter it as a duplicate if it was delivered to that destination previously.

            tabish Timothy A. Bish added a comment - The reason the Message do not reappear in the DLQ is due to Message Audit on the DLQ filtering what would be duplicate messages. I've created a test case and included it in trunk to show that if Audit is disabled on the DLQ then the Messages are moved back in. The Move operation places the message onto the destination without modifying its Message Id so any Audited resource will filter it as a duplicate if it was delivered to that destination previously.

            Working as designed. See AMQ3405Test.java in trunk for an example.

            tabish Timothy A. Bish added a comment - Working as designed. See AMQ3405Test.java in trunk for an example.

            Timothy, I have found that any message I create through the admin console will also not be sent to the DLQ, even if I set it as persistent. Do you know the cause for this?

            sbrandwoo Stephen Brandwood added a comment - Timothy, I have found that any message I create through the admin console will also not be sent to the DLQ, even if I set it as persistent. Do you know the cause for this?

            While we can work around this with disable of the Audit, it would be better to fix this so that the Audit of the Message is rolled back as messages are moved out of the DLQ.

            tabish Timothy A. Bish added a comment - While we can work around this with disable of the Audit, it would be better to fix this so that the Audit of the Message is rolled back as messages are moved out of the DLQ.

            Fixed now on trunk, when a message is moved out of a DLQ its rolled back from the DLQ's Audit so subsequent failures will result in it reappearing in the DLQ.

            tabish Timothy A. Bish added a comment - Fixed now on trunk, when a message is moved out of a DLQ its rolled back from the DLQ's Audit so subsequent failures will result in it reappearing in the DLQ.

            Commit 1ebf1862795286505dc2ce2d36e91008029061a3 in activemq's branch refs/heads/master from gtully
            [ https://git-wip-us.apache.org/repos/asf?p=activemq.git;h=1ebf186 ]

            AMQ-3405 - rework fix to use original destination strategy which may be different from the default shared strategy. new test

            jira-bot ASF subversion and git services added a comment - Commit 1ebf1862795286505dc2ce2d36e91008029061a3 in activemq's branch refs/heads/master from gtully [ https://git-wip-us.apache.org/repos/asf?p=activemq.git;h=1ebf186 ] AMQ-3405 - rework fix to use original destination strategy which may be different from the default shared strategy. new test

            Commit 509c781669fa681eded9d11681e1cb92e5f5d553 in activemq's branch refs/heads/activemq-5.15.x from gtully
            [ https://gitbox.apache.org/repos/asf?p=activemq.git;h=509c781 ]

            AMQ-3405 - rework fix to use original destination strategy which may be different from the default shared strategy. new test

            (cherry picked from commit 1ebf1862795286505dc2ce2d36e91008029061a3)

            jira-bot ASF subversion and git services added a comment - Commit 509c781669fa681eded9d11681e1cb92e5f5d553 in activemq's branch refs/heads/activemq-5.15.x from gtully [ https://gitbox.apache.org/repos/asf?p=activemq.git;h=509c781 ] AMQ-3405 - rework fix to use original destination strategy which may be different from the default shared strategy. new test (cherry picked from commit 1ebf1862795286505dc2ce2d36e91008029061a3)

            People

              tabish Timothy A. Bish
              carfey Carey Flichel
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: