[ATLAS-2634] Large Notification Messages: Avoid Processing of Already Processed Messages

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: trunk
• Fix Version/s: 1.0.0, 0.8.3
• Component/s: atlas-core
• Assignee: Ashutosh Mestry
• Reporter: Ashutosh Mestry
• Due: 04/May/18
• Created: 01/May/18 21:27
• Updated: 18/Sep/18 18:10
• Resolved: 16/May/18 21:34

Description

Scenario

If a hook encounters messages that have size larger than what Kafka can handle, it has either compresses or splits or does both to break down the message in a size that Kafka can handle.

When Atlas encounters such a message as part of processing messages from the hook, it uses appropriate strategy to get the message back in the correct format.

When a message of this type is processed, there is a possibility that the processing will go on for over the threshold mandated by Kafka for commit. If the processing exceeds the threshold, Kafka will resend that message.

This causes the message to be reprocessed. 

Given this, it is possible that the message may be stuck in the queue forever or at the very least, it is re-processed several times (at least twice).

 

Solution

• Record the message Ids for large messages.
  • For messages with no version number, calculate MD5 hash of the message and use that as message id.
• If a message with same Id is encountered again, commit the same, without processing.