Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-955

After a leader change, messages sent with ack=0 are lost

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments


    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: 0.8.0
    • Component/s: None
    • Labels:


      If the leader changes for a partition, and a producer is sending messages with ack=0, then messages will be lost, since the producer has no active way of knowing that the leader has changed, until it's next metadata refresh update.

      The broker receiving the message, which is no longer the leader, logs a message like this:

      Produce request with correlation id 7136261 from client on partition [mytopic,0] failed due to Leader not local for partition [mytopic,0] on broker 508818741

      This is exacerbated by the controlled shutdown mechanism, which forces an immediate leader change.

      A possible solution to this would be for a broker which receives a message, for a topic that it is no longer the leader for (and if the ack level is 0), then the broker could just silently forward the message over to the current leader.


        1. KAFKA-955.v1.patch
          6 kB
          Guozhang Wang
        2. KAFKA-955.v1.patch
          3 kB
          Guozhang Wang
        3. KAFKA-955.v2.patch
          6 kB
          Guozhang Wang
        4. KAFKA-955.v3.patch
          6 kB
          Guozhang Wang
        5. KAFKA-955.v4.patch
          6 kB
          Guozhang Wang
        6. KAFKA-955.v5.patch
          5 kB
          Guozhang Wang
        7. KAFKA-955.v6.patch
          9 kB
          Guozhang Wang
        8. KAFKA-955.v7.patch
          10 kB
          Guozhang Wang
        9. KAFKA-955-followup.v1.patch
          1 kB
          Guozhang Wang



            • Assignee:
              guozhang Guozhang Wang
              jbrosenberg Jason Rosenberg


              • Created:

                Issue deployment