Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-7219

Spark portable runner: support Spark metrics

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • P3
    • Resolution: Unresolved
    • None
    • None
    • runner-spark

    Description

      The Spark portable runner already updates and collects metrics from the SDK harness, but it needs to send the collected metrics on to Spark itself, similar to FlinkMetricContainer (https://github.com/apache/beam/blob/5c31efbae5fa40c024a47a44af3e6bc8a079cc2a/runners/flink/src/main/java/org/apache/beam/runners/flink/metrics/FlinkMetricContainer.java).

      Attachments

        Issue Links

          blocks

          Wish - General wishlist item. BEAM-11213 Beam metrics should be displayed in Spark UI

          • P2 - Default priority. Will be triaged and planned according to community practices.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. BEAM-4776 Java PortableRunner should support metrics

          • P2 - Default priority. Will be triaged and planned according to community practices.
          • Triage Needed

          Improvement - An improvement or enhancement to an existing feature or task. BEAM-5212 Update native spark metrics

          • P3 - Non-urgent bugs, features, and improvements
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. BEAM-9997 Improve step names in portable runners

          • P3 - Non-urgent bugs, features, and improvements
          • Open

          Activity

            People

              Unassigned Unassigned
              ibzib Kyle Weaver
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: