Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-21352 FLIP-158: Generalized incremental checkpoints
  3. FLINK-23344

State ownership: track and discard private state (materialized: rocks, heap)

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 1.15.0
    • Labels:
      None

      Description

      Inject TaskStateRegistry added in FLINK-23139 into existing state backends and use it there to manage private state.

       

      Should only take effect if changelog.enabled = true.

       

      Related task: prevent RocksDB from re-uploading state if checkpoints weren't confirmed (or were aborted).

      (see also FLINK-23461)

      Please implement or create a seperate ticket.

       

      An alternative solution that was considered is to analyze the returned snapshot for state to manage.

       

      https://docs.google.com/document/d/1NJJQ30P27BmUvD7oa4FChvkYxMEgjRPTVdO1dHLl_9I/edit#heading=h.9dxopqajsy7 

        Attachments

        Issue Links

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              roman Roman Khachatryan

              Dates

              • Created:
                Updated:

                Issue deployment