[FLINK-16536] Implement InputChannel state recovery for unaligned checkpoint

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.11.0
• Component/s: Runtime / Checkpointing, Runtime / Network
• Labels: pull-request-available
• Assignee: Zhijiang
• Reporter: Zhijiang
• Created: 11/Mar/20 05:35
• Updated: 11/Sep/24 10:33
• Resolved: 12/May/20 08:45

Description

During recovery process for unaligned checkpoint, the input channel state should also be recovered besides with existing operator states.

The InputGate would request buffer from local pool and then interact with ChannelStateReader to fill in the state data.  The filled buffer would be inserted into respective InputChannel queue for processing in normal way.

It should guarantee that the new data from upstream side should not overtake the input state data to avoid mis-order issue.

Refer to more details by https://docs.google.com/document/d/16_MOQymzxrKvUHXh6QFr2AAXIKt_2vPUf8vzKy4H_tU/edit

Issue Links

• causes: FLINK-17799 Performance regression in all network benchmarks
• causes: FLINK-36260 numBytesInLocal and numBuffersInLocal being reported as remote
• links to: GitHub Pull Request #11687