Kafka
  1. Kafka
  2. KAFKA-686

0.8 Kafka broker should give a better error message when running against 0.7 zookeeper

    Details

    • Type: Bug Bug
    • Status: Patch Available
    • Priority: Blocker Blocker
    • Resolution: Unresolved
    • Affects Version/s: 0.8.0
    • Fix Version/s: 0.8.2
    • Component/s: None
    • Labels:

      Description

      People will not know that the zookeeper paths are not compatible. When you try to start the 0.8 broker pointed at a 0.7 zookeeper you get a NullPointerException. We should detect this and give a more sane error.

      Error:
      kafka.common.KafkaException: Can't parse json string: null
      at kafka.utils.Json$.liftedTree1$1(Json.scala:20)
      at kafka.utils.Json$.parseFull(Json.scala:16)
      at kafka.utils.ZkUtils$$anonfun$getReplicaAssignmentForTopics$2.apply(ZkUtils.scala:498)
      at kafka.utils.ZkUtils$$anonfun$getReplicaAssignmentForTopics$2.apply(ZkUtils.scala:494)
      at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:61)
      at scala.collection.immutable.List.foreach(List.scala:45)
      at kafka.utils.ZkUtils$.getReplicaAssignmentForTopics(ZkUtils.scala:494)
      at kafka.controller.KafkaController.initializeControllerContext(KafkaController.scala:446)
      at kafka.controller.KafkaController.onControllerFailover(KafkaController.scala:220)
      at kafka.controller.KafkaController$$anonfun$1.apply$mcV$sp(KafkaController.scala:85)
      at kafka.server.ZookeeperLeaderElector.elect(ZookeeperLeaderElector.scala:53)
      at kafka.server.ZookeeperLeaderElector.startup(ZookeeperLeaderElector.scala:43)
      at kafka.controller.KafkaController.startup(KafkaController.scala:381)
      at kafka.server.KafkaServer.startup(KafkaServer.scala:90)
      at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:34)
      at kafka.Kafka$.main(Kafka.scala:46)
      at kafka.Kafka.main(Kafka.scala)
      Caused by: java.lang.NullPointerException
      at scala.util.parsing.combinator.lexical.Scanners$Scanner.<init>(Scanners.scala:52)
      at scala.util.parsing.json.JSON$.parseRaw(JSON.scala:71)
      at scala.util.parsing.json.JSON$.parseFull(JSON.scala:85)
      at kafka.utils.Json$.liftedTree1$1(Json.scala:17)
      ... 16 more

        Activity

        Hide
        Phil Hargett added a comment -

        While, yes, the original issue is unaddressed, there is a lurking bug in ZkUtils: a null pointer is getting wrapped in Some when it shouldn't--it's misleading. The callers of this ZkUtils method even expect that None could be returned, so there is no need to mishandle null. To be fair, the method could probably be named readDataMaybeNone, but who am I to quibble.

        It would seem like a higher-order check for 0.8 vs. 0.7 Zookeeper data is in order. That is, perhaps on startup Kafka should check the state of Zookeeper data, and report an error/warning at that time. Low-level APIs like this shouldn't necessarily have to worry about version mismatch; it could be expensive if they did. Relying upon a misleading exception (NullPointerException) rather than something that specifically diagnoses the issue would seem an incorrect choice.

        That's my $.02 after having only spent ~2 days in this code, but hope this helps.

        Show
        Phil Hargett added a comment - While, yes, the original issue is unaddressed, there is a lurking bug in ZkUtils: a null pointer is getting wrapped in Some when it shouldn't--it's misleading. The callers of this ZkUtils method even expect that None could be returned, so there is no need to mishandle null. To be fair, the method could probably be named readDataMaybeNone, but who am I to quibble. It would seem like a higher-order check for 0.8 vs. 0.7 Zookeeper data is in order. That is, perhaps on startup Kafka should check the state of Zookeeper data, and report an error/warning at that time. Low-level APIs like this shouldn't necessarily have to worry about version mismatch; it could be expensive if they did. Relying upon a misleading exception (NullPointerException) rather than something that specifically diagnoses the issue would seem an incorrect choice. That's my $.02 after having only spent ~2 days in this code, but hope this helps.
        Hide
        Swapnil Ghike added a comment -

        How do we differentiate whether the 0.8 broker is pointing at 0.7 zookeeper data, or the zookeeper data for 0.8 got messed up (using zkclient etc) by mistake? Perhaps the right solution is to rename the zookeeper directory for 0.8.

        Show
        Swapnil Ghike added a comment - How do we differentiate whether the 0.8 broker is pointing at 0.7 zookeeper data, or the zookeeper data for 0.8 got messed up (using zkclient etc) by mistake? Perhaps the right solution is to rename the zookeeper directory for 0.8.
        Hide
        Phil Hargett added a comment -

        Here's the diff as an attachment, for easier usage.

        Show
        Phil Hargett added a comment - Here's the diff as an attachment, for easier usage.
        Hide
        Phil Hargett added a comment - - edited

        While this doesn't produce a message clarifying the error, it does prevent the error in the first place. I am a novice at Scala, but I think there is a bug in ZkClient that was wrapping a null value in a Some instance--which caused the Json.parseFull to fail, as it received a null string to parse.

        This patch attempts to detect the null from Zookeeper and return a tuple with None instead of Some(null).

        Patch here using git diff

        diff --git a/core/src/main/scala/kafka/utils/ZkUtils.scala b/core/src/main/scala/kafka/utils/ZkUtils.scala
        index c6119d9..90ee2d0 100644
        — a/core/src/main/scala/kafka/utils/ZkUtils.scala
        +++ b/core/src/main/scala/kafka/utils/ZkUtils.scala
        @@ -419,7 +419,12 @@ object ZkUtils extends Logging {
        def readDataMaybeNull(client: ZkClient, path: String): (Option[String], Stat) = {
        val stat: Stat = new Stat()
        val dataAndStat = try {

        • (Some(client.readData(path, stat)), stat)
          + val data: String = client.readData(path,stat)
          + if(data != null) { + (Some(client.readData(path, stat)), stat) + }

          else

          { + (None,stat) + }

          } catch {
          case e: ZkNoNodeException =>
          (None, stat)

        Show
        Phil Hargett added a comment - - edited While this doesn't produce a message clarifying the error, it does prevent the error in the first place. I am a novice at Scala, but I think there is a bug in ZkClient that was wrapping a null value in a Some instance--which caused the Json.parseFull to fail, as it received a null string to parse. This patch attempts to detect the null from Zookeeper and return a tuple with None instead of Some(null). Patch here using git diff diff --git a/core/src/main/scala/kafka/utils/ZkUtils.scala b/core/src/main/scala/kafka/utils/ZkUtils.scala index c6119d9..90ee2d0 100644 — a/core/src/main/scala/kafka/utils/ZkUtils.scala +++ b/core/src/main/scala/kafka/utils/ZkUtils.scala @@ -419,7 +419,12 @@ object ZkUtils extends Logging { def readDataMaybeNull(client: ZkClient, path: String): (Option [String] , Stat) = { val stat: Stat = new Stat() val dataAndStat = try { (Some(client.readData(path, stat)), stat) + val data: String = client.readData(path,stat) + if(data != null) { + (Some(client.readData(path, stat)), stat) + } else { + (None,stat) + } } catch { case e: ZkNoNodeException => (None, stat)

          People

          • Assignee:
            Unassigned
            Reporter:
            Jay Kreps
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:

              Development