[ATLAS-4620] Refactor EmbeddedKafkaServer in Atlas notification module to remove core kafka dependency

• Type: Bug
• Status: In Progress
• Priority: Major
• Resolution: Unresolved
• Component/s: atlas-core
• Assignee: Patrik Márton
• Reporter: Patrik Márton
• Created: 15/Jun/22 08:48
• Updated: 31/Aug/22 09:50

Description

Goal is to break the strong coupling between Atlas components and Kafka. These dependencies include using server side libraries of Kafka (this couples the Scala version and other non-public interfaces of Kafka). Any code using server side libraries of Kafka should be refactored.

The notification module uses an EmbeddedKafkaServer which brings in the core kafka dependency. Based on the usage in the source, it is only used during testing, but pulled in as a compile dependency. 

If there is a way to avoid using this embedded kafka server for other reasons (eg. development), this can be moved to the test folder and pulled in as a test scope dependency. 

https://github.com/apache/atlas/blob/master/notification/src/main/java/org/apache/atlas/kafka/EmbeddedKafkaServer.java