[ATLAS-58] Make hive hook reliable

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.6-incubating
• Labels: incompatible
• Assignee: Shwetha GS
• Reporter: Shwetha GS
• Created: 10/Jul/15 10:13
• Updated: 09/Dec/15 14:31
• Resolved: 22/Sep/15 09:15

Description

Currently, hive hook executes in background thread pool and is an best effort approach to register entities. But this needs to be reliable for data governance to be effective

One way is - in hive hook, add the entities to some messaging framework and atlas server can read the entities from the message and register in atlas. Since, posting message is faster, we can do it synchronously and hence reliable entity registration.

We can start with kafka for messaging, but any other messaging framework should be pluggable

Issue Links

• links to: https://reviews.apache.org/r/38393/