[FLINK-19533] Add state assignment to channel state during rescaling

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.12.0
• Fix Version/s: 1.12.0
• Component/s: Runtime / Checkpointing
• Labels: pull-request-available
• Assignee: Arvid Heise
• Reporter: Arvid Heise
• Created: 08/Oct/20 06:31
• Updated: 22/Jun/21 14:07
• Resolved: 06/Nov/20 07:04

Description

Enhance StateAssignmentOperation to create a channel state mapping during rescaling, where channels before rescaling are mapped to channels after rescaling.

Example:

Before rescaling: parallelism of 2, channel 0 = key group range [0,64), channel 1 = key group range [64,128)

After rescaling: parallelism of 3, channel 0 = key group range [0,43), channel 1 = key group range [43,86), channel 2 = key group range [86,128)

Mapping: 0 -> [0, 1], 1 -> [1, 2]

 

The mapping is passed to the respective channels through deployment descriptors. It will be used in a separate ticket to restore the correct buffers from pre-rescaling channels to post-rescaling channels.

Issue Links

• links to: GitHub Pull Request #13735