Uploaded image for project: 'ActiveMQ'
  1. ActiveMQ
  2. AMQ-4952

When duplicate message occur from network producer, messages blocked by cursor audit are blocked till restart

    Details

      Description

      If auditNetworkProducers is off (as it is by default) a reforward (because of a missing send reply) will be a duplicate.
      If the initial messages is still in the store the duplicate will be trapped by the message cursor but the message remains in the store. So on a restart the cursor is fresh (unless in the case of jdbc when there are still messages in the store) and the message gets redispatched. In the jdbc case - if the store/destination is empty there will be a resend b/c there is no sequence state to replay.
      Duplicates detected by the cursor need to deal with the duplicate message - moving to the DLQ makes the most sense - the cause can indicate the reason to allow separation from poison due to redelivery failure.

      Client duplicate detection currently uses a poison ack, so this makes the behaviour consistent.

        Issue Links

          Activity

          Hide
          gtully Gary Tully added a comment -

          linking to https://issues.apache.org/jira/browse/AMQ-4465 for more context

          need to deal with duplicates to prevent eventuall delivery from https://issues.apache.org/jira/browse/AMQ-4924

          Show
          gtully Gary Tully added a comment - linking to https://issues.apache.org/jira/browse/AMQ-4465 for more context need to deal with duplicates to prevent eventuall delivery from https://issues.apache.org/jira/browse/AMQ-4924
          Hide
          gtully Gary Tully added a comment -

          when the cursor detects a duplicate the message is removed and referred for DLQ processing with a reasonCause of "duplicate from store"
          Thus it does not remain "stuck" till a restart. The typical use case where this can occur is with a duplicate forward from a network connector that fails before getting a send reply happens after the original message has been dispatched. So the store does not trap the duplicate but the cursor does.

          http://git-wip-us.apache.org/repos/asf/activemq/commit/f92d45be

          Show
          gtully Gary Tully added a comment - when the cursor detects a duplicate the message is removed and referred for DLQ processing with a reasonCause of "duplicate from store" Thus it does not remain "stuck" till a restart. The typical use case where this can occur is with a duplicate forward from a network connector that fails before getting a send reply happens after the original message has been dispatched. So the store does not trap the duplicate but the cursor does. http://git-wip-us.apache.org/repos/asf/activemq/commit/f92d45be
          Hide
          gtully Gary Tully added a comment -

          dealt with the audit false case and duplicate in pagedInMessages - now redirecting to DLQ and removing from the store in that case also
          http://git-wip-us.apache.org/repos/asf/activemq/commit/cc47ab6d

          JDBC store does not have a separate index to trap the duplicate that is still inflight. Also, cursor audit window may be too small for some use cases.
          If there is a valid usecase where duplicates are acceptable we can revisit but all tests currently pass with the change that 'outlaws' and deals with duplicates.

          Show
          gtully Gary Tully added a comment - dealt with the audit false case and duplicate in pagedInMessages - now redirecting to DLQ and removing from the store in that case also http://git-wip-us.apache.org/repos/asf/activemq/commit/cc47ab6d JDBC store does not have a separate index to trap the duplicate that is still inflight. Also, cursor audit window may be too small for some use cases. If there is a valid usecase where duplicates are acceptable we can revisit but all tests currently pass with the change that 'outlaws' and deals with duplicates.
          Hide
          gtully Gary Tully added a comment -

          further refinement in further refinement in http://git-wip-us.apache.org/repos/asf/activemq/commit/27b3a7c3

          Show
          gtully Gary Tully added a comment - further refinement in further refinement in http://git-wip-us.apache.org/repos/asf/activemq/commit/27b3a7c3

            People

            • Assignee:
              gtully Gary Tully
              Reporter:
              gtully Gary Tully
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development