Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-4687

InvalidTopicException is possible for topic colliding with itself when concurrently created


    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s:
    • Fix Version/s:
    • Component/s: admin
    • Labels:


      In our system we programmatically create the topic when data is first received from a data source prior to producing messages to that topic. Our Kafka producers run in a highly concurrent distributed environment (Storm), such that when a data feed is first enabled there may be multiple Storm bolt processes competing to create the topic. Occasionally, we see an InvalidTopicException for a topic colliding with itself due to a race condition where the initial zkUtils.zkClient.exists(topic) check returns false, but the subsequent zkUtils.getAllTopics() call includes the topic.

      Example stack trace,

      Caused by: kafka.common.InvalidTopicException: Topic "DATA.7b3fbbd4-4ecd-4521-aed4-0ccca371e573" collides with existing topics: DATA.7b3fbbd4-4ecd-4521-aed4-0ccca371e573
      	at kafka.admin.AdminUtils$.createOrUpdateTopicPartitionAssignmentPathInZK(AdminUtils.scala:258) ~[stormjar.jar:?]
      	at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:237) ~[stormjar.jar:?]
      	at kafka.admin.AdminUtils.createTopic(AdminUtils.scala) ~[stormjar.jar:?]

      To fix this I believe we just need to update the Topic#hasCollision method to disallow a topic from having a collision with itself (as KAFKA-2337 clearly did not intend for that condition to ever be possible), like this:

      def hasCollision(topicA: String, topicB: String): Boolean = {
        topicA != topicB && topicA.replace('.', '_') == topicB.replace('.', '_')

      This bug affects version and higher.


          Issue Links



              • Assignee:
                noslowerdna Andrew Olson
                noslowerdna Andrew Olson
              • Votes:
                0 Vote for this issue
                3 Start watching this issue


                • Created: