[GOBBLIN-1974] Ensure Adhoc Flows can be Executed in Multi-active Scheduler state

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: gobblin-service
• Assignee: Abhishek Tiwari
• Reporter: Urmi Mustafi
• Created: 13/Dec/23 01:33
• Updated: 14/Dec/23 20:19

Description

Adhoc flow executions delete the flow spec from the FlowCatalog after adding the spec to the scheduler for the single active scheduler. This makes sense in prior context since the single active scheduler has the information and the spec is no longer needed. However, in the multi-active case we run into a No spec found error here https://github.com/apache/gobblin/blob/00c60ca6492644652c66905eb384ab145e348bbf/gobblin-service/src/main/java/org/apache/gobblin/service/monitoring/DagActionStoreChangeMonitor.java#L263. This PR updates the code to delete the spec after persisting it in the adhoc execution case.