[AMQ-4924] Duplicate messages are left in the persistence store

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 5.8.0, 5.9.0
• Fix Version/s: 5.9.1, 5.10.0
• Component/s: Broker
• Assignee: Robert Davies
• Reporter: Ron Koerner
• Created: 06/Dec/13 19:56
• Updated: 02/Apr/14 22:19
• Resolved: 11/Dec/13 11:17

Description

We have a local and remote broker connected with a duplex bridge, which is initiated by the remote broker.
Producers are attached to the remote broker, one consumer to the local broker.
The following scenario causes messages to be left in the local store, which are replayed when the local broker is restarted:

1. messages are forwarded from the remote broker to the local broker
2. messages are dispatched to the local consumer
3. the connection between the local and remote broker fails
4. the local broker tries to acknowledge the message reception to the remote broker, which fails
5. the remote broker reconnects
6. the messages are resent
7. the local broker correctly identifies them as duplicates, but puts them into the store nevertheless where they remain until the local broker is restarted
8. other messages are produced and consumed without a problem
9. the local broker is restarted
10. the duplicates are now delivered to the local consumer again and of course out of order

This behaviour can be identified by a queue size which does not seem to shrink below a certain number, even if a consumer is connected and consuming other messages.

When the log level is set to TRACE these messages indicate the problem:

2013-12-06 20:35:17,405 TRACE .a.a.b.r.c.AbstractStoreCursor - org.apache.activemq.broker.region.cursors.QueueStorePrefetch@c0bc4f:testqueue,batchResetNeeded=false,storeHasMessages=true,size=0,cacheEnabled=true,maxBatchSize:1 - cursor got duplicate: ID:smcexp5-58011-1386358514283-7:1:1:1:1, 4 [ActiveMQ VMTransport: vm://LOCAL#19-1]
2013-12-06 20:35:17,412 TRACE .a.a.b.r.c.AbstractStoreCursor - org.apache.activemq.broker.region.cursors.QueueStorePrefetch@c0bc4f:testqueue,batchResetNeeded=false,storeHasMessages=false,size=1,cacheEnabled=false,maxBatchSize:1 - fillBatch [ActiveMQ BrokerService[LOCAL] Task-2]

Issue Links

• relates to: AMQ-4952 When duplicate message occur from network producer, messages blocked by cursor audit are blocked till restart