[GOBBLIN-1857] Allow override key for job execution ID in Helix Gobblin Cluster

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: gobblin-cluster
• Assignee: Hung Tran
• Reporter: William Lo
• Created: 19/Jul/23 20:23
• Updated: 20/Jul/23 21:18

Description

Job Execution ID is automatically inferred from the flow execution ID if the job was orchestrated by Gobblin-as-a-Service. However, this can lead to bugs in conjunction with other job keys such as earlyStop, since it would create multiple planningJobs and job IDs sent to Helix. These jobs would then have the same execution ID which is rejected by Helix.

When those configurations are set, we want to force the Gobblin cluster to default to creating the job execution ID from its timestamp.