Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-24402

Add a metric for back-pressure from the ChangelogStateBackend

    XMLWordPrintableJSON

Details

    Description

      FLINK-23381 adds back-pressure, this task is to add monitoring for that.

      See design doc: https://docs.google.com/document/d/1k5WkWIYzs3n3GYQC76H9BLGxvN3wuq7qUHJuBPR9YX0/edit#heading=h.ayt6cka7z0qf

      Can be reported as back-pressured by backend per second, similar to how "regular" back-pressure is currently reported (prototype).
      Metric name: stateBackendBlockedTimeMsPerSecond

       Take into account:

      • there is blocking and non-blocking waiting for changelog availability (see https://github.com/apache/flink/pull/17229#discussion_r740111285)
      • UI needs to be adjusted in several places: Task label; Task details
      • Back-pressure status label should probably be adjusted
      • If changelog is disabled then the metric shouldn't be shown

      Consider whether to include changelog back-pressure into overall back-pressure (https://github.com/apache/flink/pull/17229#discussion_r738322138 ).
       

       Uploading metrics should be added in FLINK-23486.

      Attachments

        Issue Links

          Activity

            People

              Yanfei Lei Yanfei Lei
              roman Roman Khachatryan
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: