[AMQ-6133] Message updates can cause message loss on recovery

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 5.13.0
• Fix Version/s: 5.13.1, 5.13.3, 5.14.0
• Component/s: Broker, KahaDB
• Assignee: Christopher L. Shannon
• Reporter: Christopher L. Shannon
• Created: 19/Jan/16 15:47
• Updated: 01/Apr/16 16:52
• Resolved: 01/Apr/16 16:52

Description

After doing some testing with AMQ-6131, I noticed a similar issue where messages can be lost if a message update is called.

Normally when KahaDB gets a KahaUpdateMessageCommand it will update the index with the new location. This works except that if the index is deleted or corrupted and needs to be rebuilt, the replay process may not be able to recover the message.

This happens because after the message is updated, KahaDB is free to garbage collect the file with the original add command. So, whatt happens is that during replay when the update command is seen KahaDB rejects it because it can't find the original message if that file has been GC'd. This happens in the updateIndex method of MessageDatabase on line 1395 where it prints out a warning saying "Non existent message update attempt rejected".

I am attaching a unit test that demonstrates the issue where the count after restart is missing messages.

Issue Links

• relates to: AMQ-6131 Durable subscription rewrote in journal can be to agressive and cause message loss on recovery