[FLINK-23344] State ownership: track and discard private state (materialized: rocks, heap)

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Won't Fix
• Fix Version/s: 1.16.0
• Component/s: Runtime / State Backends
• Assignee: Unassigned
• Reporter: Roman Khachatryan
• Created: 09/Jul/21 21:57
• Updated: 15/Aug/22 14:25
• Resolved: 15/Aug/22 14:25

Description

Inject TaskStateRegistry added in FLINK-23139 into existing state backends and use it there to manage private state.

 

Should only take effect if changelog.enabled = true.

 

Related task: prevent RocksDB from re-uploading state if checkpoints weren't confirmed (or were aborted).

(see also FLINK-23461)

Please implement or create a seperate ticket.

 

An alternative solution that was considered is to analyze the returned snapshot for state to manage.

 

https://docs.google.com/document/d/1NJJQ30P27BmUvD7oa4FChvkYxMEgjRPTVdO1dHLl_9I/edit#heading=h.9dxopqajsy7 

Issue Links

• is a clone of: FLINK-23139 State ownership: track and discard private state (registry+changelog)
• is blocked by: FLINK-23139 State ownership: track and discard private state (registry+changelog)