[GOBBLIN-2173] Adhoc flows are not being deleted from GaaS FlowSpec store

• Type: Bug
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Component/s: gobblin-service
• Assignee: Abhishek Tiwari
• Reporter: Abhishek Jain
• Created: 15/Nov/24 04:45
• Updated: 19/Nov/24 19:57

Description

In GaaS, we store adhoc flows temporarily in our flowspec DB in order to persist them in service restart/failover scenarios. However, it is expected that once these flows are kicked off/ forwarded to the DagProcEngine, they need to be removed from our flowspec db.

This is currently not consistently happening, there seems to be some edge case(s) where they are persisted in the db. This can be fatal for users such as DIL that run adhoc flows using the same flowgroup/flowname consistently, which will lead to their flows being stuck. We need to find which edge cases are not handling the flow spec deletion properly.