Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-21352 FLIP-158: Generalized incremental checkpoints
  3. FLINK-23139

State ownership: track and discard private state (registry+changelog)

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

      Description

      TM should own changelog backend state to prevent re-uploading state on checkpoint abortion (or missing confirmation). A simpler solution to only own aborted state is less maintanable in the long run.

      For that, on TM state should be tracked and discarded (on subsumption+materialization; on shutdown). 

      See state ownership design doc, in particular Tracking private state.

       

      This ticket is about creating TaskStateRegistry and using it in ChangelogStateBackend (for non-materialized part only; for materialized see FLINK-23344).
       

      Externalized checkpoints and savepoints should be supported (or please create a separate ticket).

       

      Retained checkpoints is a separate ticket: FLINK-23251

        Attachments

        Issue Links

          Activity

            People

            • Assignee:
              roman Roman Khachatryan
              Reporter:
              roman Roman Khachatryan

              Dates

              • Created:
                Updated:

                Issue deployment