Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-8360

Implement task-local state recovery

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 1.5.0
    • Runtime / State Backends
    • None

    Description

      This issue tracks the development of recovery from task-local state. The main idea is to have a secondary, local copy of the checkpointed state, while there is still a primary copy in DFS that we report to the checkpoint coordinator.

      Recovery can attempt to restore from the secondary local copy, if available, to save network bandwidth. This requires that the assignment from tasks to slots is as sticky is possible.

      For starters, we will implement this feature for all managed keyed states and can easily enhance it to all other state types (e.g. operator state) later.

      Attachments

        Issue Links

        depends upon

        Improvement - An improvement or enhancement to an existing feature or task. FLINK-7719 Send checkpoint id to task as part of deployment descriptor when resuming

        • Major - Major loss of function.
        • Closed

        Improvement - An improvement or enhancement to an existing feature or task. FLINK-7720 Centralize creation of backends and state related resources

        • Major - Major loss of function.
        • Closed
        is related to

        Improvement - An improvement or enhancement to an existing feature or task. FLINK-11159 Allow configuration whether to fall back to savepoints for restore

        • Major - Major loss of function.
        • Closed
        supercedes

        New Feature - A new feature of the product, which has yet to be developed. FLINK-7873 Introduce CheckpointCacheManager for reading checkpoint data locally when performing failover

        • Major - Major loss of function.
        • Closed
        links to

        Web Link GitHub Pull Request #5239

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            srichter Stefan Richter
            srichter Stefan Richter
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment