[ATLAS-3427] Atlas hooks enhancements for better fault-tolerance i.e. Kafka unavailability

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Assignee: Ashutosh Mestry
• Reporter: Nixon Rodrigues
• Created: 26/Sep/19 06:56
• Updated: 03/Aug/21 05:59
• Resolved: 12/Nov/20 23:20

Description

Background

Existing implementation for sending messages from the hooks relies on Kafka. Which means that if Kafka is not available for any reasons, the messages do not make it to Atlas. They are simply added to the failed.log.

 Solution

Introduce a mechanism which preserves messages if destination for the messages is not reachable. This new mechanism will also have logic to retry these messages.

Requirements

The mechanism should be:

• Transparent: Existing hooks should not have to rework their implementation. 
• Configurable: This should be introduced optionally. If hook chooses not to use this, it should fallback to existing implementation. The behavior should be configurable via properties.
• High-performance: Using this mechanism should not introduce additional overhead. Ideally, there should not be additional serialization introduced.
• Reliable: The new mechanism should relay these messages reliably. Existing message formats should be supported viz. plain, compressed, compress & multi-part.

 

Issue Links

• links to: Code Review