Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-14423

Optimize CPU bottleneck caused by metrics capture

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • network
    • None

    Description

      I ran a produce-only workload on a plaintext single broker. One of the main findings was that kafka.network.Processor.updateRequestMetrics(kafka.network.RequestChannel$Response method is expensive to the extent that 90% of the time taken by Processor.processCompletedSends() is taken by this method.

      The flamegraph is attached.

      The objective of this ticket is to optimize the performance of metrics collection here.

      Attachments

        1. flamegraph1.html
          640 kB
          Divij Vaidya
        2. Screenshot 2022-11-28 at 12.11.23.png
          296 kB
          Divij Vaidya
        3. Screenshot 2022-11-28 at 15.26.30.png
          657 kB
          Divij Vaidya

        Issue Links

          is a child of

          Improvement - An improvement or enhancement to an existing feature or task. KAFKA-14541 Profile produce workload for Apache Kafka

          • Major - Major loss of function.
          • In Progress

          Activity

            People

              Unassigned Unassigned
              divijvaidya Divij Vaidya
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: