[ATLAS-3832] Atlas notification consumer hook is not consuming from atlas_hook topic on Azure Event Hub

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.0.0
• Component/s: kafka-integration
• Labels: documentation newbie
• Flags: Important
• Assignee: Unassigned
• Reporter: Kenny
• Created: 10/Jun/20 00:20
• Updated: 10/Jun/20 03:30

Description

I have followed the instructions in the article to configure Atlas (local deployment) to point to Azure Event Hub: https://medium.com/@reenugrewal/data-lineage-tracking-using-spline-on-atlas-via-event-hub-6816be0fd5c7

I can see incoming message being sent to the atlas_hook topic, but Atlas is not consuming these messages. The application.log shows no errors. I've attached the application.log that shows successful start up of the notification consumer hook.

I'm not sure why the topic isn't being consumed by atlas and have no idea where I should be looking to troubleshoot.

I spun up a standalone zookeeper but according to the logs the atlas.kafka.zookeeper.connect isn't being used and there are no errors being thrown about it so I don't think that is a problem.

Environment setup:

• atlas 2.0.0-rc2 with embedded HBase and Solr
  • CentOS 7 with latest VirtualBox
• databricks runtime 5.5. LTS
  • spline 0.3.6 jars