Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-3173

Error while moving some partitions to OnlinePartition state

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: 0.9.0.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      We observed another instance of the problem reported in KAFKA-2300, but this time the error appeared in the partition state machine. In KAFKA-2300, we haven't cleaned up the state in PartitionStateMachine and ReplicaStateMachine as we do in KafkaController.

      Here is the stack trace:

      2016-01-29 15:26:51,393] ERROR [Partition state machine on Controller 0]: Error while moving some partitions to OnlinePartition state 
      (kafka.controller.PartitionStateMachine)java.lang.IllegalStateException: Controller to broker state change requests batch is not empty while creating a new one. 
      Some LeaderAndIsr state changes Map(0 -> Map(foo-0 -> (LeaderAndIsrInfo:
      (Leader:0,ISR:0,LeaderEpoch:0,ControllerEpoch:1),ReplicationFactor:1),AllReplicas:0))) might be lost        at 
      kafka.controller.ControllerBrokerRequestBatch.newBatch(ControllerChannelManager.scala:254)        at 
      kafka.controller.PartitionStateMachine.handleStateChanges(PartitionStateMachine.scala:144)        at 
      kafka.controller.KafkaController.onNewPartitionCreation(KafkaController.scala:517)        at 
      kafka.controller.KafkaController.onNewTopicCreation(KafkaController.scala:504)        at 
      kafka.controller.PartitionStateMachine$TopicChangeListener$$anonfun$handleChildChange$1.apply$mcV$sp(PartitionStateMachine.scala:437)        at 
      kafka.controller.PartitionStateMachine$TopicChangeListener$$anonfun$handleChildChange$1.apply(PartitionStateMachine.scala:419)        at 
      kafka.controller.PartitionStateMachine$TopicChangeListener$$anonfun$handleChildChange$1.apply(PartitionStateMachine.scala:419)        at 
      kafka.utils.CoreUtils$.inLock(CoreUtils.scala:262)        at 
      kafka.controller.PartitionStateMachine$TopicChangeListener.handleChildChange(PartitionStateMachine.scala:418)        at 
      org.I0Itec.zkclient.ZkClient$10.run(ZkClient.java:842)        at org.I0Itec.zkclient.ZkEventThread.run(ZkEventThread.java:71)
      

        Attachments

        1. KAFKA-3173-race-repro.patch
          9 kB
          Flavio Paiva Junqueira

          Issue Links

            Activity

              People

              • Assignee:
                fpj Flavio Paiva Junqueira
                Reporter:
                fpj Flavio Paiva Junqueira
                Reviewer:
                Jun Rao
              • Votes:
                0 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated: