[FLINK-29109] Checkpoint path conflict with stateless upgrade mode

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: kubernetes-operator-1.1.0
• Fix Version/s: kubernetes-operator-1.2.0, kubernetes-operator-1.3.0
• Component/s: Kubernetes Operator
• Labels: pull-request-available
• Assignee: Thomas Weise
• Reporter: Thomas Weise
• Created: 26/Aug/22 00:59
• Updated: 04/Dec/22 20:49
• Resolved: 04/Dec/22 20:49

Description

A stateful job with stateless upgrade mode (yes, there are such use cases) fails with checkpoint path conflict due to constant jobId and FLINK-19358 (applies to Flink < 1.16x). Since with stateless upgrade mode the checkpoint id resets on restart the job is going to write to previously used locations and fail. The workaround is to rotate the jobId on every redeploy when the upgrade mode is stateless. While this can be worked around externally it is best done in the operator itself because reconciliation resolves when a restart is actually required while rotating jobId externally may trigger unnecessary restarts.

Issue Links

• links to: GitHub Pull Request #385
• links to: GitHub Pull Request #469