[ATLAS-3874] NotificationHookConsumer: Shell Entity Resolution When Multiple Consumers Process Same Entity

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0, trunk
• Fix Version/s: trunk
• Component/s: atlas-core
• Assignee: Ashutosh Mestry
• Reporter: Ashutosh Mestry
• Created: 01/Jul/20 15:45
• Updated: 02/Sep/20 07:01
• Resolved: 02/Sep/20 07:01

Description

Background

Atlas has ability to consume messages from multiple hooks. When doing so, if there are messages in multiple queue requesting creation of  the same entity, then this case leads to a race condition. 

Steps to Duplicate

Pre-requisites

Atlas should be configured to consume from 2 Kafka queues viz. queue1 and queue2.

Steps:

1. Add a message in queue1 that references a non-existent entity.
2. Add a message in queue2 that creates the entity in step 1.

Expected behavior:

Shell entity should get created in step 1 that is resolved into first class entity in step 2.

Actual behavior:

Shell entity is not resolved. A duplicate entity with same qualifiedName gets created.

Root Cause

The NotificationHookConsumer is not aware of messages processed by other consumers there by leading to concurrent entity creation. 

 

Issue Links

• links to: Code Review