[FLINK-24232] Archiving of suspended jobs prevents breaks subsequent archive attempts

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.14.0, 1.13.1, 1.12.5
• Fix Version/s: 1.13.6, 1.14.3, 1.15.0
• Component/s: Runtime / Coordination
• Labels: pull-request-available
• Assignee: Chesnay Schepler
• Reporter: Chesnay Schepler
• Created: 09/Sep/21 14:10
• Updated: 16/Dec/21 20:41
• Resolved: 16/Dec/21 20:41

Description

To archive a job we write a file that uses the job ID as the name. Since suspended jobs are handled like other terminal jobs they are also being archived.

When that job then later resumes any attempt to archive the job on termination will fail because an archive already exists.

The simplest option is to add a suffix if an archive already exists, like "_1".

Issue Links

• is caused by: FLINK-22434 Dispatcher does not store suspended jobs in execution graph store
• relates to: FLINK-20195 Jobs endpoint returns duplicated jobs
• links to: GitHub Pull Request #18067