Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-9175

MirrorMaker 2 emits invalid topic partition metrics

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.4.0
    • Fix Version/s: 2.4.0
    • Component/s: KafkaConnect
    • Labels:
      None

      Description

      While looking at MirrorMaker 2 metrics with Edoardo Comar, we noticed the topic partition metrics were invalid.

      There is no traffic on the topic spp.hello but its metrics are constantly updating.

      The issue is in MirrorMetrics.PartitionMetrics. In the constructor, Sensors are built using metrics.sensor() with a name that does not include the topic partition. The method metrics.sensor() does not always create a new Sensor but can return an existing Sensor if one exists for the specified name. So in practice, if a Task is handling many topic partitions, they all share the same Sensors!

      This renders the topic partition metrics unusable and really prevents running MirrorMaker 2 in a production environment.

        Attachments

        1. image-2019-11-12-17-42-45-773.png
          223 kB
          Mickael Maison

          Activity

            People

            • Assignee:
              mimaison Mickael Maison
              Reporter:
              mimaison Mickael Maison
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: