[FLINK-11059] JobMaster may continue using an invalid slot if releasing idle slot meet a timeout

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.7.0
• Fix Version/s: 1.7.3, 1.8.1, 1.9.0
• Component/s: Runtime / Coordination
• Labels: pull-request-available
• Assignee: shuai.xu
• Reporter: shuai.xu
• Created: 04/Dec/18 03:57
• Updated: 30/Aug/19 09:51
• Resolved: 14/Jun/19 12:37

Description

When job master releases an idle slot to task executor, it may meet a timeout exception which cause that task executor may have already released the slot, but job master will add the slot back to available slots, and the slot may be used again. Then job master will continue deploying task to the slot, but task executor does not recognize it.

Issue Links

• causes: FLINK-12895 TaskManagerProcessFailureBatchRecoveryITCase.testTaskManagerProcessFailure failed on travis
• causes: FLINK-12863 Race condition between slot offerings and AllocatedSlotReport
• is duplicated by: FLINK-3539 Sync running Execution and Task instances via heartbeats
• links to: GitHub Pull Request #7227