[FLINK-21504] State ownership: notify TMs about checkpoint subsumption

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.15.0
• Component/s: Runtime / Checkpointing
• Labels: auto-unassigned pull-request-available
• Assignee: Yun Tang
• Reporter: Roman Khachatryan
• Created: 25/Feb/21 14:34
• Updated: 31/Aug/22 11:24
• Resolved: 10/Dec/21 02:31

Description

Goal: enable TMs to discard state after subsumption (FLINK-23139).

Proposed solution: include earliest non-subsumed checkpoint ID into checkpoint confirmation notification.

Code-wise, it would require passing the checkpoint ID through many existing CheckpointListener implementations (e.g. Task). CheckpointListener is a public interface and should not be concerned with retained checkpoints (runtime detail). So it's better to remove "implements" from such classes and either call methods directly or introduce a new (runtime) interface if necessary.

 

 See state ownership design doc, in particular subsumption notifications

Issue Links

• blocks: FLINK-23139 State ownership: track and discard private state (registry+changelog)
• links to: GitHub Pull Request #16582