Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-4313

ISRs may thrash when replication quota is enabled

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.1.0
    • Fix Version/s: 0.10.1.1, 0.10.2.0
    • Component/s: None
    • Labels:
      None

      Description

      I created a high volume topic and a low volume topic, each with 1000 partitions and 2 replicas on 2 brokers. I populated the high volume topic with 4GB of data. Added a third broker and ran the rebalance tool with a 2MB/sec replication quota. At some point, all replicas for the low volume topic have been moved over and the new replicas are all in sync, but the high volume replicas are still catching up. Then, I started slowly producing to the low volume topic. Once I do that, I started seeing the low volume partitions' ISR constantly expanding and shrinking. The observed ISR change rate is ~ 6/sec. So, it does seem that ISR thrashing can happen.

        Attachments

          Activity

            People

            • Assignee:
              junrao Jun Rao
              Reporter:
              junrao Jun Rao

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment