Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-2937

Topics marked for delete in Zookeeper may become undeletable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.0.0
    • Fix Version/s: 0.9.0.1
    • Component/s: core
    • Labels:
      None

      Description

      In our clusters, we occasionally see topics marked for delete, but never actually deleted. It may be due to brokers being restarted while tests were running, but further restarts of Kafka dont fix the problem. The topics remain marked for delete in Zookeeper.

      Topic describe shows:

      Topic:testtopic PartitionCount:1 ReplicationFactor:3 Configs:
      Topic: testtopic Partition: 0 Leader: none Replicas: 3,4,0 Isr:

      Kafka logs show:

      2015-12-02 15:53:30,152] ERROR Controller 2 epoch 213 initiated state change of replica 3 for partition [testtopic,0] from OnlineReplica to OfflineReplica failed (state.change.logger)
      kafka.common.StateChangeFailedException: Failed to change state of replica 3 for partition [testtopic,0] since the leader and isr path in zookeeper is empty
      at kafka.controller.ReplicaStateMachine.handleStateChange(ReplicaStateMachine.scala:269)
      at kafka.controller.ReplicaStateMachine$$anonfun$handleStateChanges$2.apply(ReplicaStateMachine.scala:114)
      at kafka.controller.ReplicaStateMachine$$anonfun$handleStateChanges$2.apply(ReplicaStateMachine.scala:114)
      at scala.collection.immutable.HashSet$HashSet1.foreach(HashSet.scala:322)
      at scala.collection.immutable.HashSet$HashTrieSet.foreach(HashSet.scala:978)
      at kafka.controller.ReplicaStateMachine.handleStateChanges(ReplicaStateMachine.scala:114)
      at kafka.controller.TopicDeletionManager$$anonfun$startReplicaDeletion$2.apply(TopicDeletionManager.scala:342)
      at kafka.controller.TopicDeletionManager$$anonfun$startReplicaDeletion$2.apply(TopicDeletionManager.scala:334)
      at scala.collection.immutable.Map$Map1.foreach(Map.scala:116)
      at kafka.controller.TopicDeletionManager.startReplicaDeletion(TopicDeletionManager.scala:334)
      at kafka.controller.TopicDeletionManager.kafka$controller$TopicDeletionManager$$onPartitionDeletion(TopicDeletionManager.scala:367)
      at kafka.controller.TopicDeletionManager$$anonfun$kafka$controller$TopicDeletionManager$$onTopicDeletion$2.apply(TopicDeletionManager.scala:313)
      at kafka.controller.TopicDeletionManager$$anonfun$kafka$controller$TopicDeletionManager$$onTopicDeletion$2.apply(TopicDeletionManager.scala:312)
      at scala.collection.immutable.Set$Set1.foreach(Set.scala:79)
      at kafka.controller.TopicDeletionManager.kafka$controller$TopicDeletionManager$$onTopicDeletion(TopicDeletionManager.scala:312)
      at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1$$anonfun$apply$mcV$sp$4.apply(TopicDeletionManager.scala:431)
      at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1$$anonfun$apply$mcV$sp$4.apply(TopicDeletionManager.scala:403)
      at scala.collection.immutable.Set$Set2.foreach(Set.scala:111)
      at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply$mcV$sp(TopicDeletionManager.scala:403)
      at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply(TopicDeletionManager.scala:397)
      at kafka.controller.TopicDeletionManager$DeleteTopicsThread$$anonfun$doWork$1.apply(TopicDeletionManager.scala:397)
      at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:262)
      at kafka.controller.TopicDeletionManager$DeleteTopicsThread.doWork(TopicDeletionManager.scala:397)
      at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63)

        Attachments

          Activity

            People

            • Assignee:
              mgharat Mayuresh Gharat
              Reporter:
              rsivaram Rajini Sivaram
            • Votes:
              1 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: