Details

      Description

      Savepoint documentation is a bit outdated regarding full data being stored in the savepoint path, not just a metadata file

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user NicoK opened a pull request:

          https://github.com/apache/flink/pull/4024

          FLINK-6782[docs] update snapshot documentation to reflect flink 1.3

          please also apply to master after accepting

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/NicoK/flink flink-6782

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/4024.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #4024


          commit 847b87d2fa3e8f0ec4dc4371bc6cffbc3f922925
          Author: Nico Kruber <nico@data-artisans.com>
          Date: 2017-05-31T08:50:39Z

          [hotfix][docs] update Checkpoint docs with correct code example

          commit 7554da3d546749d0a5aa6d31d36c03cc8556cc80
          Author: Nico Kruber <nico@data-artisans.com>
          Date: 2017-05-31T11:51:12Z

          FLINK-6782[docs] update snapshot documentation to reflect flink 1.3


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user NicoK opened a pull request: https://github.com/apache/flink/pull/4024 FLINK-6782 [docs] update snapshot documentation to reflect flink 1.3 please also apply to master after accepting You can merge this pull request into a Git repository by running: $ git pull https://github.com/NicoK/flink flink-6782 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/4024.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #4024 commit 847b87d2fa3e8f0ec4dc4371bc6cffbc3f922925 Author: Nico Kruber <nico@data-artisans.com> Date: 2017-05-31T08:50:39Z [hotfix] [docs] update Checkpoint docs with correct code example commit 7554da3d546749d0a5aa6d31d36c03cc8556cc80 Author: Nico Kruber <nico@data-artisans.com> Date: 2017-05-31T11:51:12Z FLINK-6782 [docs] update snapshot documentation to reflect flink 1.3
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user alpinegizmo commented on a diff in the pull request:

          https://github.com/apache/flink/pull/4024#discussion_r119358154

          — Diff: docs/setup/savepoints.md —
          @@ -129,11 +144,11 @@ $ bin/flink savepoint -d :savepointPath

          This disposes the savepoint stored in `:savepointPath`.

          -Note that since savepoints always go to a file system it is possible to also manually delete the savepoint via a regular file system operation. Keep in mind though that the savepoint only stores meta data that points to the actual checkpoint data. Therefore, if you manually want to delete a savepoint, you would have to include the checkpoint files as well. Since there is currently no straight forward way to figure out how a savepoint maps to a checkpoint, it is recommended to use the savepoint tool for this as described above.
          +Note that since savepoints always go to a file system it is possible to also manually delete the savepoint via a regular file system operation. Up to Flink 1.2, this way a more tedious task which was solved with this tool.
          — End diff –

          Up to Flink 1.2, this was a more tedious task which was performed with the savepoint tool.

          Show
          githubbot ASF GitHub Bot added a comment - Github user alpinegizmo commented on a diff in the pull request: https://github.com/apache/flink/pull/4024#discussion_r119358154 — Diff: docs/setup/savepoints.md — @@ -129,11 +144,11 @@ $ bin/flink savepoint -d :savepointPath This disposes the savepoint stored in `:savepointPath`. -Note that since savepoints always go to a file system it is possible to also manually delete the savepoint via a regular file system operation. Keep in mind though that the savepoint only stores meta data that points to the actual checkpoint data. Therefore, if you manually want to delete a savepoint, you would have to include the checkpoint files as well. Since there is currently no straight forward way to figure out how a savepoint maps to a checkpoint, it is recommended to use the savepoint tool for this as described above. +Note that since savepoints always go to a file system it is possible to also manually delete the savepoint via a regular file system operation. Up to Flink 1.2, this way a more tedious task which was solved with this tool. — End diff – Up to Flink 1.2, this was a more tedious task which was performed with the savepoint tool.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user EronWright commented on the issue:

          https://github.com/apache/flink/pull/4024

          Would you mind remarking as to whether it is safe to delete a savepoint after resuming a job from it?

          Show
          githubbot ASF GitHub Bot added a comment - Github user EronWright commented on the issue: https://github.com/apache/flink/pull/4024 Would you mind remarking as to whether it is safe to delete a savepoint after resuming a job from it?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user NicoK commented on the issue:

          https://github.com/apache/flink/pull/4024

          Deleting a savepoint will not affect other savepoints in case you are worried about this.
          Other than that it depends on you and what you want to do with it, e.g. keeping it in case you want to resume from it at a later point in case of failures or so.

          I did add a few notes which should make the first point clear.

          Show
          githubbot ASF GitHub Bot added a comment - Github user NicoK commented on the issue: https://github.com/apache/flink/pull/4024 Deleting a savepoint will not affect other savepoints in case you are worried about this. Other than that it depends on you and what you want to do with it, e.g. keeping it in case you want to resume from it at a later point in case of failures or so. I did add a few notes which should make the first point clear.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user alpinegizmo commented on a diff in the pull request:

          https://github.com/apache/flink/pull/4024#discussion_r119567341

          — Diff: docs/setup/savepoints.md —
          @@ -144,7 +144,7 @@ $ bin/flink savepoint -d :savepointPath

          This disposes the savepoint stored in `:savepointPath`.

          -Note that since savepoints always go to a file system it is possible to also manually delete the savepoint via a regular file system operation. Up to Flink 1.2, this way a more tedious task which was performed with the savepoint command above.
          +Note that it is possible to also manually delete a savepoint via regular file system operations without affecting other savepoints or checkpoints (recall that each savepoint is self-contained). Up to Flink 1.2, this way a more tedious task which was performed with the savepoint command above.
          — End diff –

          ... this was a more tedious ...

          Show
          githubbot ASF GitHub Bot added a comment - Github user alpinegizmo commented on a diff in the pull request: https://github.com/apache/flink/pull/4024#discussion_r119567341 — Diff: docs/setup/savepoints.md — @@ -144,7 +144,7 @@ $ bin/flink savepoint -d :savepointPath This disposes the savepoint stored in `:savepointPath`. -Note that since savepoints always go to a file system it is possible to also manually delete the savepoint via a regular file system operation. Up to Flink 1.2, this way a more tedious task which was performed with the savepoint command above. +Note that it is possible to also manually delete a savepoint via regular file system operations without affecting other savepoints or checkpoints (recall that each savepoint is self-contained). Up to Flink 1.2, this way a more tedious task which was performed with the savepoint command above. — End diff – ... this was a more tedious ...
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user alpinegizmo commented on the issue:

          https://github.com/apache/flink/pull/4024

          I read Eron's request a bit differently. Suppose you've taken a savepoint simply to manage a rescaling or redeployment. How quickly after resuming from the savepoint is it safe to delete it? Should one wait until the new job has been successfully checkpointed, for example?

          Show
          githubbot ASF GitHub Bot added a comment - Github user alpinegizmo commented on the issue: https://github.com/apache/flink/pull/4024 I read Eron's request a bit differently. Suppose you've taken a savepoint simply to manage a rescaling or redeployment. How quickly after resuming from the savepoint is it safe to delete it? Should one wait until the new job has been successfully checkpointed, for example?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user NicoK commented on the issue:

          https://github.com/apache/flink/pull/4024

          Personally, I would wait at least for another completed checkpoint before deleting a savepoint but from what I read, I would probably also always keep a savepoint around to recover from. I guess that depends on the availability needs and storage capacity that you have.
          Maybe @StefanRRichter or @StephanEwen can elaborate a bit more on best practices there and extend the docs accordingly.

          Show
          githubbot ASF GitHub Bot added a comment - Github user NicoK commented on the issue: https://github.com/apache/flink/pull/4024 Personally, I would wait at least for another completed checkpoint before deleting a savepoint but from what I read, I would probably also always keep a savepoint around to recover from. I guess that depends on the availability needs and storage capacity that you have. Maybe @StefanRRichter or @StephanEwen can elaborate a bit more on best practices there and extend the docs accordingly.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zentol commented on the issue:

          https://github.com/apache/flink/pull/4024

          +1 to merge.

          Show
          githubbot ASF GitHub Bot added a comment - Github user zentol commented on the issue: https://github.com/apache/flink/pull/4024 +1 to merge.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zentol commented on the issue:

          https://github.com/apache/flink/pull/4024

          merging.

          Show
          githubbot ASF GitHub Bot added a comment - Github user zentol commented on the issue: https://github.com/apache/flink/pull/4024 merging.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/flink/pull/4024

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/4024
          Hide
          Zentol Chesnay Schepler added a comment -

          1.3: bfba2b1d453266f28523c47682d3cfe5ae67ce74
          1.4: 60222a24123379bbbd044a6ca25817e64a9348b9

          Show
          Zentol Chesnay Schepler added a comment - 1.3: bfba2b1d453266f28523c47682d3cfe5ae67ce74 1.4: 60222a24123379bbbd044a6ca25817e64a9348b9

            People

            • Assignee:
              NicoK Nico Kruber
              Reporter:
              NicoK Nico Kruber
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development