Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-9731

Increased fetch request rate with leader selector due to HW propagation

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.4.0, 2.4.1
    • Fix Version/s: 2.6.0
    • Component/s: core
    • Labels:
      None

      Description

      KIP-392 adds high watermark propagation to followers as a means to better sync up followers HW with leader. The issue we have noticed after trying out 2.4.0 and 2.4.1 is a spike in fetch request rate in the default selector case (leader), that does not really require this high watermark propagation:

      This spike causes an increase in resource allocation (CPU) on the brokers.

      An easy solution would be to disable this propagation (at least) for the default leader selector case to improve the backward compatibility.

        Attachments

        1. image-2020-03-17-10-19-08-987.png
          174 kB
          Vahid Hashemian

          Activity

            People

            • Assignee:
              ijuma Ismael Juma
              Reporter:
              vahid Vahid Hashemian
            • Votes:
              2 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: