Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-4216

Replication Quotas: Control Leader & Follower Throttled Replicas Separately

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 0.10.1.0
    • 0.10.1.0
    • None
    • None

    Description

      The current algorithm in kafka-reassign-partitions applies a throttle to all moving replicas, be they leader-side or follower-side.

      A preferable solution would be to change the throttled replica list to specify whether the throttle applies to leader or follower. That way we can ensure that the regular replication will not be throttled.

      To do this we should change the way the throttled replica list is specified so it is spread over two separate properties. One that corresponds to the leader-side throttle, and the other that corresponds to the follower-side throttle.

      quota.leader.replication.throttled.replicas = [partId]:[replica],[partId]:[replica],[partId]:[replica]
      quota.follower.replication.throttled.replicas = [partId]:[replica],[partId]:[replica],[partId]:[replica]

      Then, when applying the throttle, the leader quota can be applied to all current replicas, and the follower quota can be applied only to the new replicas we are creating as part of the rebalance.

      Attachments

        Issue Links

          Activity

            People

              benstopford Ben Stopford
              benstopford Ben Stopford
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: