[FLINK-26490] Adjust the MaxParallelism or remove the MaxParallelism check when unnecessary.

• Type: Improvement
• Status: Open
• Priority: Not a Priority
• Resolution: Unresolved
• Component/s: Runtime / State Backends
• Labels: auto-deprioritized-major auto-deprioritized-minor pull-request-available
• Assignee: Unassigned
• Reporter: chenfengLiu
• Created: 04/Mar/22 09:41
• Updated: 13/Nov/23 07:26

Description

Since Flink introduce key group and MaxParallelism, Flink can rescale with less cost.
But when we want to update the job parallelism bigger than the MaxParallelism, it 's impossible cause there are so many MaxParallelism check that require new parallelism should not bigger than MaxParallelism. 

Actually, when an operator which don't contain keyed state, there should be no problem when update the parallelism bigger than the MaxParallelism,, cause only keyed state need MaxParallelism and key group.

So should we remove this check or auto adjust the MaxParallelism when we restore an operator state that don't contain keyed state?

It can make job restore from checkpoint easier.

Issue Links

• links to: GitHub Pull Request #19004