[FLINK-36014] Align the desired and sufficient resources definiton in Executing and WaitForResources states

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0-preview
• Component/s: Runtime / Coordination
• Labels: pull-request-available

Release Note

The new configuration `jobmanager.adaptive-scheduler.executing.resource-stabilization-timeout` for the AdaptiveScheduler was introduced. It defines a duration for which the JobManager delays the scaling operation after a resource change if only sufficient resources are available.

The existing configuration `jobmanager.adaptive-scheduler.min-parallelism-increase` was deprecated and is not used by Flink anymore.

See Flink's configuration documentation for further details.

• Assignee: Zdenek Tison
• Reporter: Zdenek Tison
• Created: 08/Aug/24 14:22
• Updated: 16/Sep/24 09:34
• Resolved: 11/Sep/24 06:57

Description

The goal is to use the same definition for the desired and sufficient resources in the Executing state as in the WaitingForResources state. 

Issue Links

• causes: FLINK-36279 AdaptiveScheduler#hasDesiredResources doesn't rely on all available slots which causes problems in Executing state
• links to: GitHub Pull Request #25307