Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-7284

Producer getting fenced may cause Streams to shut down

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.11.0.3, 1.0.2, 1.1.1, 2.0.0
    • Fix Version/s: 0.11.0.4, 1.0.3, 1.1.2, 2.0.1, 2.1.0
    • Component/s: streams
    • Labels:
      None

      Description

      As part of the investigation, I will determine what other versions are affected.

       

      In StreamTask, we catch a `ProducerFencedException` and throw a `TaskMigratedException`. However, in this case, the `RecordCollectorImpl` is throwing a `StreamsException`, caused by `KafkaException` caused by `ProducerFencedException`.

      In response to a TaskMigratedException, we would rebalance, but when we get a StreamsException, streams shuts itself down.

      In other words, we intended to do a rebalance in response to a producer fence, but actually, we shut down (when the fence happens inside the record collector).

      Coincidentally, Guozhang noticed and fixed this in a recent PR: https://github.com/apache/kafka/pull/5428/files#diff-4e5612eeba09dabf30d0b8430f269ff6

       

      The scope of this ticket is to extract that fix and associated tests, and send a separate PR to trunk and 2.0, and also to determine what other versions, if any, are affected.

        Attachments

          Activity

            People

            • Assignee:
              vvcephei John Roesler
              Reporter:
              vvcephei John Roesler
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: