Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-6554

Broker doesn't reject Produce request with inconsistent state

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.1.0
    • Component/s: producer
    • Labels:
      None

      Description

      Produce messages of type v3 have offset deltas in each record along with a LastOffsetDelta for the topic/partition set. In investigating an issue with missing offsets, I found a bug in a producer library where it would send multiple records, but leave LastOffsetDelta at 0. This causes various problems including holes in the offsets fetched by the consumer. 

      As lastOffsetDelta can be computed by looking at the records, it seems like the broker should at least validate the LastOffsetDelta field against the contained records to stop this bad data getting in.

      I've attached a decode v3 produce message that was causing the problems, and was accepted by the broker.

      Here's a link to the issue in the kafka library we were using which has more context if you need it.

      https://github.com/Shopify/sarama/issues/1032

       

       

        Attachments

        1. produce_v3.txt
          1 kB
          Simon Fell

          Issue Links

            Activity

              People

              • Assignee:
                hachikuji Jason Gustafson
                Reporter:
                superfell Simon Fell
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: