[FLINK-23475] Handle BroadcastState with finished operators

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.14.0
• Component/s: Runtime / Checkpointing
• Labels: pull-request-available
• Assignee: Yun Gao
• Reporter: Dawid Wysakowicz
• Created: 22/Jul/21 14:05
• Updated: 12/Aug/21 02:25
• Resolved: 12/Aug/21 02:25

Description

BroadcastState assumes all subtasks state is the same. When rescaling it reassigns states of all tasks to new tasks. If the parallelism is different it assigns the old states to new range of states in a round robin fashion.

We must make sure that an empty state is not reassigned to running subtasks in org.apache.flink.runtime.checkpoint.RoundRobinOperatorStateRepartitioner#repartitionBroadcastState

Issue Links

• links to: GitHub Pull Request #16714