Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-16553

Missing KafkaFetcher topic/partition metrics

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None

      Description

      When using the Kafka universal connector, currently not all KafkaFetcher metrics (link) which are exposed through the KafkaConsumer are accessible within the Flink metrics system.

      Especially, all metrics which are related to topics and partitions are not available. The KafkaConsumer internally only registers those metrics after it has fetched some records.

      Unfortunately, at the moment Flink only checks the available metrics right after the initialization of the KafkaConsumer when no records are polled, yet.

        Attachments

          Activity

            People

            • Assignee:
              app-tarush Tarush Grover
              Reporter:
              fabian.paul Fabian Paul
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated: