Affects Version/s: None
Fix Version/s: None
This request comes out of working with the state processor APIs.
I was originally trying to have this workflow when using the state processing APIs:
- Download savepoint / checkpoint from s3 to my pc using non-flink tooling (e.g. s3 cli)
- Process the savepoint / checkpoint locally without s3 access
This ended up not being possible.
It seems that the state processing APIs look at the checkpoint / savepoint metadata file first. Then they follow fully qualified s3 links from the metadata file. So, even though I've downloaded the entire checkpoint / savepoint to my pc, s3 access is still required.
This is a specific instance of the problem, which I think more generally can be stated as: Checkpoints/savepoints can't be moved (easily)
It would be great if savepoints / checkpoints could be moved around in a self-contained manner and used at a later time.