Kafka
  1. Kafka
  2. KAFKA-379

TopicCount.constructTopicCount isn't thread-safe

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.7, 0.8.0
    • Fix Version/s: 0.7, 0.8.0
    • Component/s: clients
    • Labels:

      Description

      TopicCount uses scala.util.parsing.json.JSON, which isn't thread-safe https://issues.scala-lang.org/browse/SI-4929

      If you have multiple consumers within the same JVM, and they all rebalance at the same time, you can get errors like the following:

      [...] kafka.consumer.TopicCount$.constructTopicCount:39] ERROR: error parsing consumer json string [...]
      java.lang.NullPointerException
      at scala.util.parsing.combinator.Parsers$NoSuccess.<init>(Parsers.scala:131)
      at scala.util.parsing.combinator.Parsers$Failure.<init>(Parsers.scala:158)
      at scala.util.parsing.combinator.Parsers$$anonfun$acceptIf$1.apply(Parsers.scala:489)
      ...
      at scala.util.parsing.combinator.Parsers$$anon$2.apply(Parsers.scala:742)
      at scala.util.parsing.json.JSON$.parseRaw(JSON.scala:71)
      at scala.util.parsing.json.JSON$.parseFull(JSON.scala:85)
      at kafka.consumer.TopicCount$.constructTopicCount(TopicCount.scala:32)
      at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.kafka$consumer$ZookeeperConsumerConnector$ZKRebalancerListener$$getTopicCount(ZookeeperConsumerConnector.scala:422)
      at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.kafka$consumer$ZookeeperConsumerConnector$ZKRebalancerListener$$rebalance(ZookeeperConsumerConnector.scala:460)
      at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener$$anonfun$syncedRebalance$1.apply$mcVI$sp(ZookeeperConsumerConnector.scala:437)
      at scala.collection.immutable.Range$ByOne$class.foreach$mVc$sp(Range.scala:282)
      at scala.collection.immutable.Range$$anon$2.foreach$mVc$sp(Range.scala:265)
      at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.syncedRebalance(ZookeeperConsumerConnector.scala:433)
      at kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.handleChildChange(ZookeeperConsumerConnector.scala:375)
      at org.I0Itec.zkclient.ZkClient$7.run(ZkClient.java:568)
      at org.I0Itec.zkclient.ZkEventThread.run(ZkEventThread.java:71)

      I ran into this on 0.7.0, but the code in trunk appears to be vulnerable to the same issue.

      1. TopicCount.scala.diff
        2 kB
        Nick Howard
      2. kafka-379_0.8_v1.patch
        8 kB
        Jun Rao

        Activity

        Nick Howard created issue -
        Nick Howard made changes -
        Field Original Value New Value
        Attachment TopicCount.scala.diff [ 12533891 ]
        Jun Rao made changes -
        Attachment kafka-379_0.8_v1.patch [ 12538804 ]
        Joel Koshy made changes -
        Labels bugs
        Jun Rao made changes -
        Assignee Jun Rao [ junrao ]
        Jun Rao made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Jun Rao made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 0.8 [ 12317244 ]
        Fix Version/s 0.7 [ 12317243 ]
        Resolution Fixed [ 1 ]
        Jun Rao made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Tony Stevenson made changes -
        Workflow no-reopen-closed, patch-avail [ 12709042 ] Apache Kafka Workflow [ 13052827 ]
        Tony Stevenson made changes -
        Workflow Apache Kafka Workflow [ 13052827 ] no-reopen-closed, patch-avail [ 13054933 ]

          People

          • Assignee:
            Jun Rao
            Reporter:
            Nick Howard
          • Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development