[FLINK-24380] Flink should handle the state transition of the pod from Pending to Failed

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.14.0, 1.13.2
• Fix Version/s: 1.13.3, 1.14.3, 1.15.0
• Component/s: Runtime / Coordination
• Labels: pull-request-available
• Assignee: Yangze Guo
• Reporter: Yangze Guo
• Created: 27/Sep/21 03:08
• Updated: 15/Dec/21 01:44
• Resolved: 29/Sep/21 07:54

Description

In K8s, there is five phases in pod's lifecycle: Pending, Running, Secceeded, Failed and Unknown. Currently, Flink does not handle the state transition of the pod from Pending to Failed. If a pod failed from Pending by `OutOfCPU` or `OutOfMem`, it will never be released and Flink keep waiting for it.

To fix this issue, Flink should terminate the pod in Failed phase proactively.

Issue Links

• links to: GitHub Pull Request #17361