[FLINK-21648] FLIP-151: Incremental snapshots for heap-based state backend

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Fix Version/s: 1.20.0
• Component/s: Runtime / State Backends
• Labels: auto-deprioritized-major auto-unassigned stale-assigned
• Assignee: Roman Khachatryan
• Reporter: Roman Khachatryan
• Created: 07/Mar/21 19:14
• Updated: 11/Mar/24 12:43

Description

Umbrella ticket for https://cwiki.apache.org/confluence/display/FLINK/FLIP-151%3A+Incremental+snapshots+for+heap-based+state+backend

 

Issue Links

• is related to: FLINK-26853 HeapStateBackend ignores metadata updates in certain cases
• mentioned in: Page Loading...

Sub-Tasks

• Allow HeapBackend snapshotting customization: In Progress
• Allow HeapBackend RestoreOperation customization: In Progress
• Allow HeapBackend creation customization: In Progress
• Allow extension of CopyOnWrite state classes: In Progress
• Notify CheckpointStrategy about checkpoint completion/abortion: In Progress
• Allow reuse of PeriodicMaterializationManager: In Progress
• Fix race condition in CopyOnWriteStateMap: In Progress
• Bump CopyOnWriteStateMap entry version before write: In Progress
• Implement incremental checkpoints: Open
• On heap state restore, skip key groups from other state backends: Resolved
• Add extension points to heap backend to read/write incremental snapshots: Resolved