[FLINK-30238] Unified Sink committer does not clean up state on final savepoint

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Invalid
• Affects Version/s: 1.17.0, 1.15.3, 1.16.1
• Component/s: Connectors / Common
• Assignee: Unassigned
• Reporter: Fabian Paul
• Created: 29/Nov/22 11:01
• Updated: 11/Apr/24 21:54
• Resolved: 11/Apr/24 21:54

Description

During stop-with-savepoint the committer only commits the pending committables on notifyCheckpointComplete.

This has several downsides.

• Last committableSummary has checkpoint id LONG.MAX and is never cleared from the state leading to that stop-with-savepoint does not work when the pipeline recovers from a savepoint 
• While the committables are committed during stop-with-savepoint they are not forwarded to post-commit topology, potentially losing data and preventing to close open transactions.