[FLINK-19385] Channel recovery may deadlock

• Type: Sub-task
• Status: Closed
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: 1.12.0
• Fix Version/s: 1.12.0
• Component/s: Runtime / Network, Runtime / Task
• Labels: pull-request-available
• Release Note: Fixed by FLINK-18990
• Assignee: Roman Khachatryan
• Reporter: Roman Khachatryan
• Created: 23/Sep/20 18:27
• Updated: 22/Jun/21 13:55
• Resolved: 09/Oct/20 17:36

Description

Consider the following case:

• Two IntputGates
• Input selection is not ALL (say FIRST initially)
• Unaligned Checkpoints ON
• on recovery, there are "parts" of records in all channels (actually 1 is enough I think)

What happens:

1. StreamTask initiates recovery and scedule partition request upon it's end
2. All gates and channels will receive buffers from StateReader
3. All channels of a single gate will consume those state buffers - completing that gate's StateConsumedFuture
4. InputProcessor will return NOTHING_AVAILABLE (see StreamTwoInputProcessor.getInputStatus)
5. StreamTask will suspend its default action
6. State of the 2nd gate won't be consumed - so its StateConsumedFutures won't be completed - so no partitions will be requested

Solution: request partitions independently for each channel.

Issue Links

• blocks: FLINK-18990 Optimize reading of channel state on recovery
• links to: GitHub Pull Request #13467