[FLINK-23772] Double check if non-keyed FullyFinishedOperatorState can be mixed up with non finished OperatorState on recovery

• Type: Sub-task
• Status: Closed
• Priority: Critical
• Resolution: Not A Problem
• Affects Version/s: 1.14.0
• Fix Version/s: 1.14.0
• Component/s: Runtime / Checkpointing
• Assignee: Unassigned
• Reporter: Piotr Nowojski
• Created: 13/Aug/21 19:24
• Updated: 01/Sep/21 10:06
• Resolved: 20/Aug/21 05:51

Description

I'm not sure if with non-keyed state we have an issue that it can be reshuffled to different operators during recovery. Are there any guarantees that if subtask 1 has state A, while subtask 2 has B, that after recovery it won’t be rotated?

1. is this an issue?
2. if so, if we have partially finished tasks with some operators having, FullyFinishedOperatorState, what prevents VerticesFinishedCache.calculateIfFinished from failing if the FullyFinishedOperatorState gets assigned to an operator chain with non finished operator?

For example an operator chain with parallelism of two, non-keyed, before recovery:

src1 (finished state) -> foo1 (finished state)
src2 -> foo2

Can we end up after recovery with:

src1 (finished state) -> foo2
src2 -> foo1 (finished state)

?