Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-5053 Incremental / lightweight snapshots for checkpoints
  3. FLINK-6034

Add KeyedStateHandle for the snapshots in keyed streams

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.3.0
    • Labels:
      None

      Description

      Currently, the only type of the snapshots in keyed streams is KeyGroupsStateHandle which is full and store the states one group after another. With the introduction of incremental checkpointing, we need a higher level abstraction of keyed snapshots to allow flexible snapshot formats.

      The implementation of KeyedStateHandle s may vary a lot in different backends. The only information needed in KeyedStateHandle s is their key group range. When recovering the job with a different degree of parallelism, KeyedStateHandle s will be assigned to those subtasks whose key group ranges overlap with their ranges.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                shixg Xiaogang Shi
                Reporter:
                shixg Xiaogang Shi
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: