[KAFKA-4914] Partition re-assignment tool should check types before persisting state in ZooKeeper

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.10.1.1
• Fix Version/s: 2.0.0
• Component/s: admin
• Assignee: Nick Travers
• Reporter: Nick Travers
• Reviewer: Ismael Juma
• Created: 17/Mar/17 03:25
• Updated: 25/Apr/18 09:42
• Resolved: 31/Mar/18 14:58

Description

The partition-reassignment too currently allows non-type-safe information to be persisted into ZooKeeper, which can result in a ClassCastException at runtime for brokers.

Specifically, this occurred when the broker assignment field was a List of Strings, instead of a List of Integers.

2017-03-15 01:44:04,572 ERROR [ZkClient-EventThread-36-samsa-zkserver.stage.sjc1.square:26101/samsa] controller.ReplicaStateMachine$BrokerChangeListener - [BrokerChangeListener on Controller 10]: Error while handling broker changes
java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Integer
        at scala.runtime.BoxesRunTime.unboxToInt(BoxesRunTime.java:101)
        at kafka.controller.KafkaController$$anonfun$8$$anonfun$apply$2.apply(KafkaController.scala:436)
        at scala.collection.LinearSeqOptimized$class.exists(LinearSeqOptimized.scala:93)
        at scala.collection.immutable.List.exists(List.scala:84)
        at kafka.controller.KafkaController$$anonfun$8.apply(KafkaController.scala:436)
        at kafka.controller.KafkaController$$anonfun$8.apply(KafkaController.scala:435)
        at scala.collection.TraversableLike$$anonfun$filterImpl$1.apply(TraversableLike.scala:248)
        at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
        at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:99)
        at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:230)
        at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
        at scala.collection.mutable.HashMap.foreach(HashMap.scala:99)
        at scala.collection.TraversableLike$class.filterImpl(TraversableLike.scala:247)
        at scala.collection.TraversableLike$class.filter(TraversableLike.scala:259)
        at scala.collection.AbstractTraversable.filter(Traversable.scala:104)
        at kafka.controller.KafkaController.onBrokerStartup(KafkaController.scala:435)
        at kafka.controller.ReplicaStateMachine$BrokerChangeListener$$anonfun$handleChildChange$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ReplicaStateMachine.scala:374)
        at kafka.controller.ReplicaStateMachine$BrokerChangeListener$$anonfun$handleChildChange$1$$anonfun$apply$mcV$sp$1.apply(ReplicaStateMachine.scala:358)
        at kafka.controller.ReplicaStateMachine$BrokerChangeListener$$anonfun$handleChildChange$1$$anonfun$apply$mcV$sp$1.apply(ReplicaStateMachine.scala:358)
        at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
        at kafka.controller.ReplicaStateMachine$BrokerChangeListener$$anonfun$handleChildChange$1.apply$mcV$sp(ReplicaStateMachine.scala:357)
        at kafka.controller.ReplicaStateMachine$BrokerChangeListener$$anonfun$handleChildChange$1.apply(ReplicaStateMachine.scala:356)
        at kafka.controller.ReplicaStateMachine$BrokerChangeListener$$anonfun$handleChildChange$1.apply(ReplicaStateMachine.scala:356)
        at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:234)
        at kafka.controller.ReplicaStateMachine$BrokerChangeListener.handleChildChange(ReplicaStateMachine.scala:355)
        at org.I0Itec.zkclient.ZkClient$10.run(ZkClient.java:843)
        at org.I0Itec.zkclient.ZkEventThread.run(ZkEventThread.java:71)

Issue Links

• Blocked: KAFKA-2218 reassignment tool needs to parse and validate the json
• duplicates: KAFKA-3033 Reassigning partition stuck in progress
• duplicates: KAFKA-4190 kafka-reassign-partitions does not report syntax problem in json
• relates to: KAFKA-6084 ReassignPartitionsCommand should propagate JSON parsing failures
• relates to: KAFKA-6413 ReassignPartitionsCommand#parsePartitionReassignmentData() should give better error message when JSON is malformed
• links to: GitHub Pull Request #2708
• links to: GitHub Pull Request #2708