[FLINK-25872] Restoring from non-changelog checkpoint with changelog state-backend enabled in CLAIM mode discards state in use

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.16.0
• Component/s: Runtime / Checkpointing, Runtime / State Backends
• Labels: pull-request-available
• Assignee: Yanfei Lei
• Reporter: Yun Tang
• Created: 28/Jan/22 17:56
• Updated: 19/Jun/23 07:15
• Resolved: 30/Jun/22 15:32

Description

If we restore from checkpoint with changelog state-backend enabled in snapshot CLAIM mode, the restored checkpoint would be discarded on subsume. This invalidates newer/active checkpoints because their materialized part is discarded (for incremental wrapped checkpoints, their private state is discarded). This bug is like FLINK-25478.

 

Design doc: https://docs.google.com/document/d/1KSFWc0gL7HkhC-JNrnsp06TLnsTmZOTHITQDcGMo0cI/edit?usp=sharing,

Issue Links

• causes: FLINK-26079 [Changelog] Disallow recovery from non-changelog checkpoints
• causes: FLINK-28597 Empty checkpoint folders not deleted on job cancellation if their shared state is still in use
• causes: FLINK-28647 Remove separate error handling and adjust documentation for CLAIM mode + RocksDB native savepoint
• is related to: FLINK-28843 Fail to find incremental handle when restoring from changelog checkpoint in claim mode
• links to: GitHub Pull Request #19448
• links to: GitHub Pull Request #22799