Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-21352

FLIP-158: Generalized incremental checkpoints

    XMLWordPrintableJSON

Details

    Description

      Umbrella ticket for FLIP-158: Generalized incremental checkpoints (v1).

      [v2] FLIP-158: Generalized incremental checkpoints can be found FLINK-25842

      Attachments

        Issue Links

          1.
          Same materialized state handle should not register multi times Sub-task Resolved Yun Tang
          2.
          Randomize periodic materialisation interval in tests Sub-task Resolved Roman Khachatryan
          3.
          Add ITCase for Generalized incremental checkpoints Sub-task Resolved Hangxiang Yu
          4.
          Add ChangelogBackend documentation Sub-task Resolved Roman Khachatryan
          5.
          Include changelog jars into distribution Sub-task Resolved Roman Khachatryan
          6.
          Prevent JM from discarding state on checkpoint abortion Sub-task Resolved Roman Khachatryan
          7.
          Write metadata after materialization Sub-task Closed Yuan Mei
          8.
          State ownership: notify TMs about checkpoint subsumption Sub-task Resolved Yun Tang
          9.
          Provide backpressure (currently job fails if a limit is hit) Sub-task Resolved Roman Khachatryan
          10.
          Add metrics for the Changelog uploader Sub-task Resolved Roman Khachatryan
          11.
          Add periodic materialization Sub-task Closed Yuan Mei
          12.
          Revert FLINK-23738 (i.e. unhide config, API, docs) Sub-task Closed Roman Khachatryan
          13.
          Hide any configuration, API or docs Sub-task Closed Roman Khachatryan
          14.
          Enable changelog backend in tests Sub-task Closed Roman Khachatryan
          15.
          Add percentiles to checkpoint stats Sub-task Closed Roman Khachatryan
          16.
          Cleanup unnecessary dependencies in dstl pom.xml Sub-task Closed Roman Khachatryan
          17.
          Optimize writing state changes Sub-task Closed Roman Khachatryan
          18.
          Add FS-based StateChangelog implementation Sub-task Closed Roman Khachatryan
          19.
          Add StateChangelog interface and its in-memory implementation Sub-task Closed Roman Khachatryan
          20.
          Provide StateBackends access to MailboxExecutor Sub-task Closed Yuan Mei
          21.
          Add ChangelogStateBackend (proxy-everything) Sub-task Closed Yuan Mei
          22.
          Send changes to the state changelog (still proxy everything) Sub-task Closed Roman Khachatryan
          23.
          Implement incremental checkpointing and recovery using state changelog Sub-task Closed Roman Khachatryan
          24.
          Test Changelog State backend (Wrapper) Sub-task Resolved Yangyang ZHANG
          25.
          Add Changelog state for timers (PQ) Sub-task Closed Piotr Nowojski
          26.
          Create and wire changelog writer with backend Sub-task Closed Zakelly Lan
          27.
          Log metadata Sub-task Closed Roman Khachatryan
          28.
          ChangelogStateBackend tests use nested backend on recovery Sub-task Closed Roman Khachatryan
          29.
          Adjust configuration of ChangelogStateBackend and hide it from users Sub-task Resolved Zakelly Lan
          30.
          Python API for enabling ChangelogStateBackend Sub-task Closed Zakelly Lan
          31.
          testKeyGroupedInternalPriorityQueue does not dispose ChangelogDelegateEmbeddedRocksDB properly, and fails the test Sub-task Closed Yun Tang
          32.
          Release TaskManagerJobMetricGroup with the last slot rather than task Sub-task Resolved Roman Khachatryan
          33.
          Introduce incremental/full checkpoint size stats Sub-task Resolved Yun Tang

          Activity

            People

              roman Roman Khachatryan
              roman Roman Khachatryan
              Votes:
              0 Vote for this issue
              Watchers:
              23 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: