[FLINK-27675] Improve manual savepoint tracking

• Type: Improvement
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: kubernetes-operator-1.0.0
• Component/s: Kubernetes Operator
• Labels: pull-request-available
• Assignee: Matyas Orhidi
• Reporter: Gyula Fora
• Created: 18/May/22 05:13
• Updated: 24/Nov/22 01:02
• Resolved: 19/May/22 13:23

Description

There are 2 problems with the manual savpeoint result observing logic that can cause the reconciler to not make progress with the deployment (recoveries, upgrades etc).

1. Whenever the jobmanager deployment is not in READY state or the job itself is not RUNNING, the trigger info must be reset and we should not try to query it anymore. Flink will not retry the savepoint if the job fails, restarted anyways.
2. If there is a sensible error when fetching the savepoint status (such as: 
   There is no savepoint operation with triggerId=xxx for job ) we should simply reset the trigger. These errors will never go away on their own and will simply cause the deployment to get stuck in observing/waiting for a savepoint to complete

Issue Links

• links to: GitHub Pull Request #225