Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-5877

Controller should only update reassignment znode if there is change in the reassignment data

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.1.0
    • Component/s: None
    • Labels:
      None

      Description

      I encountered a scenario where controller keeps printing the following stack trace repeatedly for a finite set of partitions. Although I have not fully figured out the cause of this event, it seems that controller will update the reassignment znode even if the new data is same as existing data. This patch optimizes the controller behavior by only updating reassignment znode if it needs to change the reassignment znode data.

      2017/09/12 20:34:05.842 [KafkaController] [Controller 1376005]: Error completing reassignment of partition [FederatorResultEvent,202]
      kafka.common.KafkaException: Partition [FederatorResultEvent,202] to be reassigned is already assigned to replicas 1367001,1384010,1386010. Ignoring request for partition reassignment
      at kafka.controller.KafkaController.initiateReassignReplicasForTopicPartition(KafkaController.scala:608) ~[kafka_2.10-0.11.0.9.jar:?]
      at kafka.controller.KafkaController$PartitionReassignment$$anonfun$process$14.apply(KafkaController.scala:1327) ~[kafka_2.10-0.11.0.9.jar:?]
      at kafka.controller.KafkaController$PartitionReassignment$$anonfun$process$14.apply(KafkaController.scala:1320) ~[kafka_2.10-0.11.0.9.jar:?]
      at scala.collection.immutable.HashMap$HashMap1.foreach(HashMap.scala:224) ~[scala-library-2.10.4.jar:?]
      at scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:403) ~[scala-library-2.10.4.jar:?]
      at scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:403) ~[scala-library-2.10.4.jar:?]
      at kafka.controller.KafkaController$PartitionReassignment.process(KafkaController.scala:1320) ~[kafka_2.10-0.11.0.9.jar:?]
      at kafka.controller.ControllerEventManager$ControllerEventThread$$anonfun$doWork$1.apply$mcV$sp(ControllerEventManager.scala:53) ~[kafka_2.10-0.11.0.9.jar:?]
      at kafka.controller.ControllerEventManager$ControllerEventThread$$anonfun$doWork$1.apply(ControllerEventManager.scala:53) ~[kafka_2.10-0.11.0.9.jar:?]
      at kafka.controller.ControllerEventManager$ControllerEventThread$$anonfun$doWork$1.apply(ControllerEventManager.scala:53) ~[kafka_2.10-0.11.0.9.jar:?]
      at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:31) ~[kafka_2.10-0.11.0.9.jar:?]
      at kafka.controller.ControllerEventManager$ControllerEventThread.doWork(ControllerEventManager.scala:52) ~[kafka_2.10-0.11.0.9.jar:?]
      at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:64) ~[kafka_2.10-0.11.0.9.jar:?]

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                lindong Dong Lin
                Reporter:
                lindong Dong Lin
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: