[FLINK-30669] Update recent job status in FlinkDeployment resource object.

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: kubernetes-operator-1.4.0
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Mohemmad Zaid Khan
• Created: 13/Jan/23 04:26
• Updated: 31/Jan/23 08:24
• Resolved: 31/Jan/23 08:24

Description

User jar has code as  -

main() {
 init env
 pipelines.foreach{
  env.fromSource(pipeline.getSource())
     .map(pipeline.transform())
     .sinkTo(pipeline.getSink())
  env.execute(pipeline.getName())
 }
}

and below configuration -

execution.runtime-mode: "BATCH"
execution.attached: "true"
$internal.pipeline.job-id: "" 

When this single jar executed in Application Mode by using flink-kubernetes-operator, multiple jobs are submitted sequentially and as per design only one of the JobStatus is always associated with FlinkDeployment k8s resource, this job status is periodically updated by operator. To update job status in k8s resource, it fetches all of the job status from job-manager rest endpoint and pick the first one and update that one. Problem is, job status list returned by job-manager rest api is not sorted on time.

As you can see in above example, job autoscaling-3 is first one in the rest response and same updated in FlinkDeployment resource, but FlinkDeployment should have status of job autoscaling-19 because that is the last job finished.

Issue Links

• links to: GitHub Pull Request #504