[FLINK-20195] Jobs endpoint returns duplicated jobs

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.11.2
• Fix Version/s: 1.13.6, 1.14.3, 1.15.0
• Component/s: Runtime / Coordination, Runtime / REST
• Labels: pull-request-available
• Assignee: Chesnay Schepler
• Reporter: Ingo Bürk
• Created: 17/Nov/20 16:52
• Updated: 10/Feb/22 08:31
• Resolved: 12/Dec/21 11:21

Description

The GET /jobs endpoint can, for a split second, return a duplicated job after it has been cancelled. This occurred in Ververica Platform after canceling a job (using PATCH /jobs/{jobId}) and calling GET /jobs.

I've reproduced this and queried the endpoint in a relatively tight loop (~ every 0.5s) to log the responses of GET /jobs and got this:

 

 

…

{"jobs":[{"id":"e110531c08dd4e3dbbfcf7afc1629c3d","status":"RUNNING"},{"id":"53fd11db25394308862c997dce9ef990","status":"CANCELLING"}]}

{"jobs":[{"id":"e110531c08dd4e3dbbfcf7afc1629c3d","status":"RUNNING"},{"id":"53fd11db25394308862c997dce9ef990","status":"CANCELLING"}]}

{"jobs":[{"id":"e110531c08dd4e3dbbfcf7afc1629c3d","status":"FAILED"},{"id":"53fd11db25394308862c997dce9ef990","status":"CANCELED"},{"id":"53fd11db25394308862c997dce9ef990","status":"CANCELED"}]}

{"jobs":[{"id":"53fd11db25394308862c997dce9ef990","status":"CANCELED"},{"id":"e110531c08dd4e3dbbfcf7afc1629c3d","status":"FAILED"}]}

{"jobs":[{"id":"53fd11db25394308862c997dce9ef990","status":"CANCELED"},{"id":"e110531c08dd4e3dbbfcf7afc1629c3d","status":"FAILED"}]}

…

 

You can see in in between that for just a moment, the endpoint returned the same Job ID twice.

 

Issue Links

• causes: FLINK-25732 Dispatcher#requestMultipleJobDetails returns non-serialiable collection
• is duplicated by: FLINK-24594 /jobs endpoint returns same job id twice (suspended and running) until stopped
• is related to: FLINK-24232 Archiving of suspended jobs prevents breaks subsequent archive attempts
• links to: GitHub Pull Request #18078