Kafka
  1. Kafka
  2. KAFKA-822 Reassignment of partitions needs a cleanup
  3. KAFKA-821

specifying a non-existent partition to ./bin/kafka-reassign-partitions.sh breaks all reassignment ops

    Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: None
    • Component/s: controller, tools
    • Labels:

      Description

      From my ML post...the workaround is to manually delete /admin/reassign_partitions from ZK, that node contained a json with an empty partitions array.

      Have 3 brokers running. Ids 25,26,27

      ./bin/kafka-create-topic.sh --replica 3 --topic first-cluster-topic
      --zookeeper <zk>:2181/kafka

      Seems fine, can send/receive, etc..

      Kill 27, start 28. Try to reassign the single partition topic with the
      following json.

      Contains an error. partition should be 0 not 1.

      {"partitions":
      [

      {"topic": "first-cluster-topic", "partition": 1, "replicas": [25,26,28] }

      ]
      }

      ./bin/kafka-reassign-partitions.sh --zookeeper ... -path-to-json-file
      reassign.json

      2013-03-21 12:14:46,170] INFO zookeeper state changed (SyncConnected)
      (org.I0Itec.zkclient.ZkClient)
      [2013-03-21 12:14:46,310] ERROR Skipping reassignment of partition
      [first-cluster-topic,1] since it doesn't exist
      (kafka.admin.ReassignPartitionsCommand)
      Successfully started reassignment of partitions Map([first-cluster-topic,1]
      -> List(25, 26, 28))
      [2013-03-21 12:14:46,665] INFO Terminate ZkClient event thread.
      (org.I0Itec.zkclient.ZkEventThread)
      [2013-03-21 12:14:46,780] INFO Session: 0x13d8a63a3760007 closed
      (org.apache.zookeeper.ZooKeeper)
      [2013-03-21 12:14:46,780] INFO EventThread shut down
      (org.apache.zookeeper.ClientCnxn)

      Ok, fix the JSON

      {"partitions":
      [

      {"topic": "first-cluster-topic", "partition": 0, "replicas": [25,26,28] }

      ]
      }

      ./bin/kafka-reassign-partitions.sh --zookeeper ... -path-to-json-file
      reassign.json

      [2013-03-21 12:17:34,367] INFO zookeeper state changed (SyncConnected)
      (org.I0Itec.zkclient.ZkClient)
      Partitions reassignment failed due to Partition reassignment currently in
      progress for Map(). Aborting operation
      kafka.common.AdminCommandFailedException: Partition reassignment currently
      in progress for Map(). Aborting operation
      at
      kafka.admin.ReassignPartitionsCommand.reassignPartitions(ReassignPartitionsCommand.scala:91)
      at
      kafka.admin.ReassignPartitionsCommand$.main(ReassignPartitionsCommand.scala:65)
      at
      kafka.admin.ReassignPartitionsCommand.main(ReassignPartitionsCommand.scala)

      ./bin/kafka-check-reassignment-status.sh --zookeeper ...
      --path-to-json-file reassign.json

      [2013-03-21 12:20:40,607] INFO zookeeper state changed (SyncConnected)
      (org.I0Itec.zkclient.ZkClient)
      Exception in thread "main" java.lang.ClassCastException:
      scala.collection.immutable.Map$Map1 cannot be cast to
      [Lscala.collection.Map;
      at
      kafka.admin.CheckReassignmentStatus$.main(CheckReassignmentStatus.scala:44)
      at kafka.admin.CheckReassignmentStatus.main(CheckReassignmentStatus.scala)

        Activity

          People

          • Assignee:
            Swapnil Ghike
            Reporter:
            Scott Clasen
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development