Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-7040

The replica fetcher thread may truncate accepted messages during multiple fast leadership transitions

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: In Progress
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: replication
    • Labels:
      None

      Description

      Problem Statement:
      Consider the scenario where there are two brokers, broker0, and broker1, and there are two partitions "t1p0", and "t1p1"[1], both of which have broker1 as the leader and broker0 as the follower. The following sequence of events happened on broker0

      1. The replica fetcher thread on a broker0 issues a LeaderEpoch request to broker1, and awaits to get the response
      2. A LeaderAndISR request causes broker0 to become the leader for one partition t1p0, which in turn will remove the partition t1p0 from the replica fetcher thread
      3. Broker0 accepts some messages from a producer
      4. A 2nd LeaderAndISR request causes broker1 to become the leader, and broker0 to become the follower for partition t1p0. This will cause the partition t1p0 to be added back to the replica fetcher thread on broker0.
      5. The replica fetcher thread on broker0 receives a response for the LeaderEpoch request issued in step 1, and truncates the accepted messages in step3.

      The issue can be reproduced with the test from https://github.com/gitlw/kafka/commit/8956e743f0e432cc05648da08c81fc1167b31bea

      [1] Initially we set up broker0 to be the follower of two partitions instead of just one, to avoid the shutting down of the replica fetcher thread when it becomes idle.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                apovzner Anna Povzner
                Reporter:
                luwang Lucas Wang
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated: