[FLINK-34563] Autoscaling decision improvement

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: kubernetes-operator-1.7.0
• Component/s: Kubernetes Operator
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Yang LI
• Created: 01/Mar/24 17:20
• Updated: 12/Mar/24 11:54

Description

Hi, I'd like to propose a minor improvement based on my autoscaling experiments. The concept revolves around identifying the vertex with the highest level of parallelism and matching it to the maximum parallelism supported by our task manager.

The primary goal of this enhancement is to prevent any task slots from remaining unused after the Flink autoscaler performs a rescaling operation. I've already tested this modification in a custom build of the operator, excluding the memory tuning feature. However, I believe it could be beneficial, especially in scenarios where the memory tuning feature is not enabled.

And I have prepared this small pr also

Issue Links

• links to: GitHub Pull Request #787