[FLINK-17560] No Slots available exception in Apache Flink Job Manager while Scheduling

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Abandoned
• Affects Version/s: 1.8.3
• Component/s: Runtime / Coordination
• Labels: auto-deprioritized-major

Environment

Flink verson 1.8.3

Session cluster

• Assignee: Unassigned
• Reporter: josson paul kalapparambath
• Created: 07/May/20 21:15
• Updated: 30/Apr/21 09:04
• Resolved: 30/Apr/21 09:04

Description

Set up
------
Flink verson 1.8.3

Zookeeper HA cluster

1 ResourceManager/Dispatcher (Same Node)
1 TaskManager
4 pipelines running with various parallelism's

Issue
------

Occationally when the Job Manager gets restarted we noticed that all the pipelines are not getting scheduled. The error that is reporeted by the Job Manger is 'not enough slots are available'. This should not be the case because task manager was deployed with sufficient slots for the number of pipelines/parallelism we have.

We further noticed that the slot report sent by the taskmanger contains solts filled with old CANCELLED job Ids. I am not sure why the task manager still holds the details of the old jobs. Thread dump on the task manager confirms that old pipelines are not running.

I am aware of https://issues.apache.org/jira/browse/FLINK-12865. But this is not the issue happening in this case.

Issue Links

• is related to: FLINK-18293 TaskExecutor offering non empty slots can lead to resource violation