[GOBBLIN-1997] Avoid deleting flowSpec too early

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: gobblin-service
• Assignee: Abhishek Tiwari
• Reporter: Urmi Mustafi
• Created: 30/Jan/24 18:25
• Updated: 30/Jan/24 19:07

Description

https://github.com/apache/gobblin/blob/67c610117c2b757bb15714826e0b0edc3e6b26a5/gobblin-service/src/main/java/org/apache/gobblin/service/monitoring/DagActionStoreChangeMonitor.java#L290 

If a non-active host receives and processes the LAUNCH event before the leader does, then we encounter a SpecNotFoundException when the leader goes to process the event and the execution of the LAUNCH will be missed. This PR moves the deletion to the DagManager and does it only after completing addDag to persist the event and after checking active status.