Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-8328

Kafka smooth expansion

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 0.10.2.0
    • 0.10.2.0
    • clients, core
    • Patch

    Description

      When expanding the kafka cluster, the new follower will read the data from the earliest offset. This can result in a large amount of historical data being read from the disk, putting a lot of pressure on the disk and affecting the performance of the kafka service, for example, the producer write latency will increase. In general, kafka's original expansion mechanism has the following problems:
                1. The new follower will put a lot of pressure on the disk;
                2. Causes the producer write latency to increase;
                3. Causes the consumer read latency to increase;
             In order to solve these problems, we have proposed a solution for smooth expansion. The main idea of the scheme is that the newly added follower reads data from the HW position, and when the newly added follower reads the data to a certain time threshold or data size threshold, the follower enters the ISR queue. . Since the new follower reads data from the HW location, most of the data read is in the operating system's cache, so it does not put pressure on the disk and does not affect the performance of the kafka service, thus solving the above problems. 
             In order to illustrate the problems of the original expansion scheme, we have done some tests, and there are corresponding test charts in the attachment.

       

       

      Attachments

        1. DiskUtil.png
          596 kB
          Fake
        2. Kafka_smooth_expansion.patch
          66 kB
          Fake
        3. producerP999.png
          511 kB
          Fake

        Issue Links

          Activity

            People

              Unassigned Unassigned
              LordChen Fake
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated: