[FLINK-27132] CheckpointResourcesCleanupRunner might discard shared state of the initial checkpoint

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.15.0, 1.16.0
• Component/s: Runtime / Checkpointing
• Assignee: Unassigned
• Reporter: Roman Khachatryan
• Created: 08/Apr/22 08:09
• Updated: 11/Apr/22 22:39

Description

When considering the following case: # A job starts from a checkpoint in NO_CLAIM mode, with incremental checkpoints enabled

1. It produces some new checkpoints and subsumes the original one (not discarding shared state - before FLINK-24611 or after FLINK-26985)
2. Job terminates abruptly
3. The cleaner is started for that job
4. ZK doesn't have the initial checkpoint, so the store will load only the new checkpoints (created in 2). Shared state is registered
5. The store is shut down - discarding all the checkpoints and also any shared state

In 6 5, if some checkpoint uses the initial state, it will also be discarded
 
mapohl could you please confirm this?
 
cc: yunta

Issue Links

• relates to: FLINK-27114 On JM restart, the information about the initial checkpoints can be lost
• relates to: FLINK-26985 With legacy restore mode, incremental checkpoints would be deleted by mistake