Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-4775

JobService should support returning metrics

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: P2
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: Not applicable
    • Component/s: beam-model
    • Labels:
      None

      Description

      Design doc: https://s.apache.org/get-metrics-api.

      Further discussion is ongoing on this doc.

      We want to report job metrics back to the portability harness from the runner harness, for displaying to users.

      Relevant PRs in flight:

      Ready for Review:

      Iterating / Discussing:

      • #7971: Flink portable metrics: get ptransform from MonitoringInfo, not stage name
        • this is a simpler, Flink-specific PR that is basically duplicated inside each of the following two, so may be worth trying to merge in first
      • #7915: use MonitoringInfo data model in Java SDK metrics
      • #7868: MonitoringInfo URN tweaks

      Merged

      • #8018: add job metrics RPC protos
      • #7867: key MetricResult by a MetricKey
      • #7938: move MonitoringInfo protos to model/pipeline module
      • #7883: Add MetricQueryResults.allMetrics() helper
      • #7866: move function helpers from fn-harness to sdks/java/core
      • #7890: consolidate MetricResult implementations

      Closed

      • #7934: job metrics RPC + SDK support
      • #7876: Clean up metric protos; support integer distributions, gauges

        Attachments

          Activity

            People

            • Assignee:
              ŁukaszG Lukasz Gajowy
              Reporter:
              jkff Eugene Kirpichov
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 55h 20m
                55h 20m