Uploaded image for project: 'ActiveMQ'
  1. ActiveMQ
  2. AMQ-5507

"Duplicate dispatch on connection" with more than 2^31 messages

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 5.8.0, 5.10.0
    • Fix Version/s: 5.11.0
    • Component/s: Broker
    • Labels:
      None

      Description

      A few times over the last few years the busiest queue in our system would suddenly start spewing these "Duplicate dispatch on connection" warning messages into the log and 0 messages would be processed. Googling around the only references I could find mentioned "optimize acknowledge" may be at fault. We stopped using that feature and I though perhaps this issue was gone but it popped up again today.

      The log messages look like this:

      2015-01-05 20:25:21,310 [ActiveMQ Session Task-2341         ] WARN  apache.activemq.ActiveMQMessageConsumer - Duplicate dispatch on connection: ID:perf4-flexapp-43114-1419911988672-9:1 to consumer: ID:perf4-flexapp-43114-1419911988672-9:1:1:1, ignoring (auto acking) duplicate: MessageDispatch {commandId = 0, responseRequired = false, consumerId = ID:perf4-flexapp-43114-1419911988672-9:1:1:1, destination = queue://*********, message = ActiveMQBytesMessage {commandId = -497935582, responseRequired = false, messageId = ID:perf4-nc-49600-1419436870976-1:8:1:1:3797031711, originalDestination = null, originalTransactionId = null, producerId = ID:perf4-nc-49600-1419436870976-1:8:1:1, destination = queue://*********, transactionId = null, expiration = 0, timestamp = 1420507504378, arrival = 0, brokerInTime = 1420507504527, brokerOutTime = 1420507521253, correlationId = null, replyTo = null, persistent = true, type = null, priority = 4, groupID = null, groupSequence = 0, targetConsumerId = null, compressed = false, userID = null, content = org.apache.activemq.util.ByteSequence@477f9261, marshalledProperties = null, dataStructure = null, redeliveryCounter = 0, size = 0, properties = null, readOnlyProperties = true, readOnlyBody = true, droppable = false} ActiveMQBytesMessage{ bytesOut = null, dataOut = null, dataIn = null }, redeliveryCounter = 0}
      

      I randomly noticed that the "commandId" was negative. I looked at the messageId and sure enough it is very large, in the 2^31 range. I checked all my field reports for this bug and in all cases the id was just over 2^31. I dug into the activemq code and found the issue was the BitArrayBin class doesn't handle values of that size correctly. I was about to post that as a critical bug but found it had already been reported and fixed for 5.11.0 in another context. AMQ-5016

      In case other people have a system stable enough to generate 2^31 messages from a single producer they should know they can add the setting "?checkForDuplicates=false" at the queue or connection level. Are there any other mitigation strategies I may not know?

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                shendley Sam hendley
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: