[FLINK-26147] Incremental savepoints directory in CLAIM mode is not being deleted

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.15.0
• Component/s: Runtime / Checkpointing
• Assignee: Unassigned
• Reporter: Piotr Nowojski
• Created: 15/Feb/22 08:59
• Updated: 15/Feb/22 11:51

Description

Native format supports incremental RocksDB savepoints. For those savepoints Flink puts all SST files inside the savepoints directory. This means such savepoints are self-contained and relocatable. However, when restored in CLAIM mode, subsequent checkpoints might reuse some SST files, which in turn might block deleting the savepoints directory at the time the savepoint is subsumed. Later on Flink will delete the reused shared SST files, but it won’t retry deleting the savepoints directory. Therefore, it is possible Flink leaves an empty savepoints directory if it was restored in CLAIM mode.

We should find a way to clean up that left over directory.

Issue Links

• is related to: FLINK-25276 FLIP-203: Support native and incremental savepoints