[FLINK-14357] Request for the ability to move checkpoints/savepoints

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Duplicate
• Assignee: Unassigned
• Reporter: Mitch Wasson
• Created: 09/Oct/19 16:35
• Updated: 13/Apr/21 20:30
• Resolved: 11/Oct/19 12:56

Description

This request comes out of working with the state processor APIs.

I was originally trying to have this workflow when using the state processing APIs:

1. Download savepoint / checkpoint from s3 to my pc using non-flink tooling (e.g. s3 cli)
2. Process the savepoint / checkpoint locally without s3 access

This ended up not being possible.

It seems that the state processing APIs look at the checkpoint / savepoint metadata file first. Then they follow fully qualified s3 links from the metadata file. So, even though I've downloaded the entire checkpoint / savepoint to my pc, s3 access is still required.

This is a specific instance of the problem, which I think more generally can be stated as: Checkpoints/savepoints can't be moved (easily)

 

It would be great if savepoints / checkpoints could be moved around in a self-contained manner and used at a later time.

Issue Links

• duplicates: FLINK-5763 Make savepoints self-contained and relocatable