[FLINK-26985] With legacy restore mode, incremental checkpoints would be deleted by mistake

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.15.0, 1.16.0
• Fix Version/s: 1.15.0, 1.16.0
• Component/s: Runtime / Checkpointing
• Labels: pull-request-available
• Assignee: Roman Khachatryan
• Reporter: Yun Tang
• Created: 01/Apr/22 10:19
• Updated: 09/Apr/22 08:02
• Resolved: 09/Apr/22 08:02

Description

Before flink-1.15, the restored checkpoint would be regsiterd and not discard on subsume, which means the restored incremental checkpoint would have one more reference counting to avoid discard.

However, after state registry refactored, we could delete artificats in the restored incremental checkpoint with legacy restore mode.

The error could be reproduced via ResumeCheckpointManuallyITCase#testExternalizedIncrementalRocksDBCheckpointsStandalone in my local branch.

Thanks for masteryhx who found this problem in the manual test.

Issue Links

• is related to: FLINK-27132 CheckpointResourcesCleanupRunner might discard shared state of the initial checkpoint
• relates to: FLINK-27114 On JM restart, the information about the initial checkpoints can be lost
• links to: BP-1.15
• links to: GitHub Pull Request #19331