[FLINK-28597] Empty checkpoint folders not deleted on job cancellation if their shared state is still in use

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.16.0
• Fix Version/s: 1.16.0
• Component/s: Runtime / Checkpointing
• Labels: pull-request-available
• Assignee: Roman Khachatryan
• Reporter: Roman Khachatryan
• Created: 18/Jul/22 23:41
• Updated: 03/Aug/22 11:00
• Resolved: 03/Aug/22 11:00

Description

After FLINK-25872, SharedStateRegistry registers all state handles, including private ones.
Once the state isn't use AND the checkpoint is subsumed, it will actually be discarded.

This is done to prevent premature deletion when recovering in CLAIM mode:
1. RocksDB native savepoint folder (shared state is stored in chk-xx folder so it might fail the deletion)
2. Initial non-changelog checkpoint when switching to changelog-based checkpoints (private state of the initial checkpoint might be included into later checkpoints and its deletion would invalidate them)

Additionally, checkpoint folders are not deleted for a longer time which might be confusing.
In case of a crash, more folders will remain.

cc: Yanfei Lei, ym

Issue Links

• is caused by: FLINK-25872 Restoring from non-changelog checkpoint with changelog state-backend enabled in CLAIM mode discards state in use
• is related to: FLINK-28581 Test Changelog StateBackend V2 Manually
• links to: GitHub Pull Request #20300
• links to: GitHub Pull Request #20313