Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-7190

Under low traffic conditions purging repartition topics cause WARN statements about UNKNOWN_PRODUCER_ID

    Details

    • Type: Improvement
    • Status: Reopened
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.1.0, 1.1.1
    • Fix Version/s: None
    • Component/s: core, streams
    • Labels:
      None

      Description

      When a streams application has little traffic, then it is possible that consumer purging would delete
      even the last message sent by a producer (i.e., all the messages sent by
      this producer have been consumed and committed), and as a result, the broker
      would delete that producer's ID. The next time when this producer tries to
      send, it will get this UNKNOWN_PRODUCER_ID error code, but in this case,
      this error is retriable: the producer would just get a new producer id and
      retries, and then this time it will succeed. 

       

      Possible fixes could be on the broker side, i.e., delaying the deletion of the produderIDs for a more extended period or on the streams side developing a more conservative approach to deleting offsets from repartition topics

       

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                guozhang Guozhang Wang
                Reporter:
                bbejeck Bill Bejeck
              • Votes:
                3 Vote for this issue
                Watchers:
                15 Start watching this issue

                Dates

                • Created:
                  Updated: