Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-6170

Some checkpoint metrics rely on latest stat snapshot

    Details

      Description

      Some checkpoint metrics use the latest stats snapshot to get the returned metric value. These snapshots are only updated when the WebRuntimeMonitor actually requests some stats (web UI or REST API).

      In practice, this means that these metrics are only updated when users are browsing the web UI.

      Instead of relying on the latest snapshot, the checkpoint metrics should be directly updated via the completion callbacks.

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user uce opened a pull request:

          https://github.com/apache/flink/pull/3597

          FLINK-6170 [metrics] Don't rely on stats snapshot for checkpoint metrics

          Some checkpoint metrics use the latest stats snapshot to get the returned metric value. In practice, this means that these metrics are only updated when users are browsing the web UI (only then is a new snapshot created).

          Instead of relying on the latest snapshot, the checkpoint metrics for the latest completed checkpoint should be directly updated when a checkpoint completes.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/uce/flink 6170-checkpoint_metrics_update

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/3597.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #3597



          Show
          githubbot ASF GitHub Bot added a comment - GitHub user uce opened a pull request: https://github.com/apache/flink/pull/3597 FLINK-6170 [metrics] Don't rely on stats snapshot for checkpoint metrics Some checkpoint metrics use the latest stats snapshot to get the returned metric value. In practice, this means that these metrics are only updated when users are browsing the web UI (only then is a new snapshot created). Instead of relying on the latest snapshot, the checkpoint metrics for the latest completed checkpoint should be directly updated when a checkpoint completes. You can merge this pull request into a Git repository by running: $ git pull https://github.com/uce/flink 6170-checkpoint_metrics_update Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3597.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3597
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user aljoscha commented on the issue:

          https://github.com/apache/flink/pull/3597

          LGTM!

          Show
          githubbot ASF GitHub Bot added a comment - Github user aljoscha commented on the issue: https://github.com/apache/flink/pull/3597 LGTM!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user uce commented on the issue:

          https://github.com/apache/flink/pull/3597

          Thanks, will merge after Travis gives a green light.

          Show
          githubbot ASF GitHub Bot added a comment - Github user uce commented on the issue: https://github.com/apache/flink/pull/3597 Thanks, will merge after Travis gives a green light.
          Hide
          uce Ufuk Celebi added a comment -

          Fixed in d0695c0 (master), 7fbb115 (release-1.2).

          Show
          uce Ufuk Celebi added a comment - Fixed in d0695c0 (master), 7fbb115 (release-1.2).
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/flink/pull/3597

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/3597

            People

            • Assignee:
              uce Ufuk Celebi
              Reporter:
              uce Ufuk Celebi
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development