Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-4216

Replication Quotas: Control Leader & Follower Throttled Replicas Separately

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.10.1.0
    • Fix Version/s: 0.10.1.0
    • Component/s: None
    • Labels:
      None

      Description

      The current algorithm in kafka-reassign-partitions applies a throttle to all moving replicas, be they leader-side or follower-side.

      A preferable solution would be to change the throttled replica list to specify whether the throttle applies to leader or follower. That way we can ensure that the regular replication will not be throttled.

      To do this we should change the way the throttled replica list is specified so it is spread over two separate properties. One that corresponds to the leader-side throttle, and the other that corresponds to the follower-side throttle.

      quota.leader.replication.throttled.replicas = [partId]:[replica],[partId]:[replica],[partId]:[replica]
      quota.follower.replication.throttled.replicas = [partId]:[replica],[partId]:[replica],[partId]:[replica]

      Then, when applying the throttle, the leader quota can be applied to all current replicas, and the follower quota can be applied only to the new replicas we are creating as part of the rebalance.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                benstopford Ben Stopford
                Reporter:
                benstopford Ben Stopford
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: