Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-14357

Request for the ability to move checkpoints/savepoints

    XMLWordPrintableJSON

    Details

    • Type: Wish
    • Status: Closed
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      This request comes out of working with the state processor APIs.

      I was originally trying to have this workflow when using the state processing APIs:

      1. Download savepoint / checkpoint from s3 to my pc using non-flink tooling (e.g. s3 cli)
      2. Process the savepoint / checkpoint locally without s3 access

      This ended up not being possible.

      It seems that the state processing APIs look at the checkpoint / savepoint metadata file first. Then they follow fully qualified s3 links from the metadata file. So, even though I've downloaded the entire checkpoint / savepoint to my pc, s3 access is still required.

      This is a specific instance of the problem, which I think more generally can be stated as: Checkpoints/savepoints can't be moved (easily)

       

      It would be great if savepoints / checkpoints could be moved around in a self-contained manner and used at a later time.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                mlmitch Mitch Wasson
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: