[FLINK-18753] Support local recovery for Unaligned checkpoints

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 1.11.0
• Component/s: Runtime / Checkpointing, Runtime / Task
• Labels: auto-deprioritized-major
• Assignee: Unassigned
• Reporter: Roman Khachatryan
• Created: 29/Jul/20 11:18
• Updated: 05/May/23 10:26

Description

There is standard mechanism of writing two duplicating streams with state data: to DFS and to local FS. On recovery, if local state is present and it matches state on JM then it will be used (instead of downloading state from DFS).

Currently, local recovery is not supported if running with Unaligned checkpoints enabled.

To enable it, we need to

1. write this second stream (see CheckpointStreamWithResultProvider.getCheckpointOutputStream)
2. make sure localState is included into the snapshot reported to JM