Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-5419

Taskmanager metrics not accessible via REST

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.2.0, 1.3.0
    • Fix Version/s: 1.2.0, 1.3.0
    • Component/s: Metrics, Webfrontend
    • Labels:
      None

      Description

      There is currently a URL clash between the TaskManagersHandler and TaskManagerMetricsHandler, with both being routed to

      /taskmanagers/:taskmanagerid/metrics
      

      As a result it is not possible to query the full set of metrics for a taskmanager, but only the hard-coded subset that is displayed on the metrics tab on the taskmanager page.

      This is a side-effect of 6d53bbc4b92e651786ecc8c2c6dfeb8e450a16a3 making the URL's more consistent. The TaskManager page in the web-interface has 3 tabs: Metrics, Log and Stdout.

      The URLs for these tabs are

      /taskmanager/<tmid>/metrics
      /taskmanager/<tmid>/log
      /taskmanager/<tmid>/stdout
      

      which correspond to the REST URL's used. Previously, the metrics tab used

      /taskmanager/<tmid>

      However, 70704de0c82cbb7b143dd696221e11999feb3600 then exposed the metrics gathered by the metrics system through the REST API. The assumption was that general information for the taskmanagers are retrieved via /taskmanager/<tmid>, similar to how the job-related URL's are structured, which sadly isn't the case.

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user zentol opened a pull request:

          https://github.com/apache/flink/pull/3092

          FLINK-5419 Make full TM metrics available through REST API

          This PR fixes the full set of TaskManager metrics not being available through the REST API. The issue was that multiple handlers were registered for the same URL ("/taskmanagers/:taskmanagerid/metrics").
          The relevant handlers are the ```TaskManagersHandler``` which provides a subset of metrics to the WebInterface (basically what is displayed in the metrics tab in the TM page) and the ```TaskManagerMetricsHandler``` which provides full access to metrics.

          The issue was fixed by making the metrics tab use the following URL: "/taskmanagers/:taskmanagerid". This is in fact a reversal of a previous change, which technically changed a public API.

          While fixing this i found a bug in the ```TaskManagerMetricsHandler```; it used the wrong constant to extract the TaskManagerID from the path ("tmid" instead of "taskmanagerid")

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/zentol/flink 5419_tm_rest

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flink/pull/3092.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #3092


          commit d88dc84100d6d53626c3cb13c5caff5a7bfacc55
          Author: zentol <chesnay@apache.org>
          Date: 2017-01-11T13:07:03Z

          FLINK-5419 Make full TM metrics available through REST API


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user zentol opened a pull request: https://github.com/apache/flink/pull/3092 FLINK-5419 Make full TM metrics available through REST API This PR fixes the full set of TaskManager metrics not being available through the REST API. The issue was that multiple handlers were registered for the same URL ("/taskmanagers/:taskmanagerid/metrics"). The relevant handlers are the ```TaskManagersHandler``` which provides a subset of metrics to the WebInterface (basically what is displayed in the metrics tab in the TM page) and the ```TaskManagerMetricsHandler``` which provides full access to metrics. The issue was fixed by making the metrics tab use the following URL: "/taskmanagers/:taskmanagerid". This is in fact a reversal of a previous change, which technically changed a public API. While fixing this i found a bug in the ```TaskManagerMetricsHandler```; it used the wrong constant to extract the TaskManagerID from the path ("tmid" instead of "taskmanagerid") You can merge this pull request into a Git repository by running: $ git pull https://github.com/zentol/flink 5419_tm_rest Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/3092.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3092 commit d88dc84100d6d53626c3cb13c5caff5a7bfacc55 Author: zentol <chesnay@apache.org> Date: 2017-01-11T13:07:03Z FLINK-5419 Make full TM metrics available through REST API
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user StephanEwen commented on the issue:

          https://github.com/apache/flink/pull/3092

          +1 to merge this

          Looks like the web UI files need to be rebuilt when merging this...

          Show
          githubbot ASF GitHub Bot added a comment - Github user StephanEwen commented on the issue: https://github.com/apache/flink/pull/3092 +1 to merge this Looks like the web UI files need to be rebuilt when merging this...
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zentol commented on the issue:

          https://github.com/apache/flink/pull/3092

          yes, I will rebuild the files while merging. I have other commits pending that require rebuilding, and figured i could do it in one swoop

          Show
          githubbot ASF GitHub Bot added a comment - Github user zentol commented on the issue: https://github.com/apache/flink/pull/3092 yes, I will rebuild the files while merging. I have other commits pending that require rebuilding, and figured i could do it in one swoop
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/flink/pull/3092

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/3092
          Hide
          Zentol Chesnay Schepler added a comment -

          master: c93e04c16a7865dbbc90d89799b46f91831e0a55
          1.2: f90edc483000db42caaff0de650c1f5bca11bf47

          Show
          Zentol Chesnay Schepler added a comment - master: c93e04c16a7865dbbc90d89799b46f91831e0a55 1.2: f90edc483000db42caaff0de650c1f5bca11bf47

            People

            • Assignee:
              Zentol Chesnay Schepler
              Reporter:
              Zentol Chesnay Schepler
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development