Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-9653

Add operator name to latency metrics

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Runtime / Metrics
    • Labels:
      None
    • Environment:

      All

      Description

      Currently the latency metrics reports latency between subtasks using this format: {{flink_taskmanager_job_latency_source_id_source_subtask_index_operator_id_operator_subtask_index_latency{ host="",instance="",job="", job_id="",job_name="",operator_id="",operator_subtask_index="0", quantile="0.99",source_id="",source_subtask_index="0",tm_id=""}}

      The request is to add operator_name along with operator_id to the metric labels.

      For a simple job, (e.g., source->map->sink)   you can see two sets of latency metrics. Each set shows all quantiles like (.5, .95..). Only thing different between the two sets is the operator_id. This makes sense assuming one operator_id belongs to the map operator and the other belongs to the sink.

       Now the problem is that is no intuitive way to distinguish between the two (find out which operator_id is the map vs sink), just by looking at the metrics. 

      Assigning names to map and sink operator does not help. Even though these names show up in other metrics like numRecordsIn, the names does not show up in the latency metric.

      The feature request is to add operator_name along with operator_id to the metric label so that it can be easily used in dashboards and tracking.

       

       

        Attachments

        Issue Links

          Activity

            People

            • Assignee:
              yanghua vinoyang
              Reporter:
              jstephen Julian Stephen

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment