Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-1180

WhiteList topic filter gets a NullPointerException on complex Regex


    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s:
    • Component/s: consumer
    • Labels:


      We are needing to create a stream selector that essentially combines the logic of the BlackList and WhiteList classes (which is not easily exposed in the high-level consumer api). That is, we want to select a topic that contains a certain prefix, as long as it doesn't also contain a secondary string.

      This should be easy to do with ordinary java Regex's, but we're running into some issues, trying to do this with the WhiteList class only.

      We have a pattern that uses negative lookahead, like this:


      So this should select a topic like: "test-good", but exclude a topic like "test-bad", and also exclude a topic without the "test" prefix, like "foo-bar".

      Instead, what we see is a NullPointerException in the call to createMessageStreamsByFilter (after having previously sent a message to "test-good" followed by a message to "test-bad"):

      21700 [ConsumerFetcherThread-group1_square-1a7ac0.local-1386869343370-dc19c7dc-0-1946108683] ERROR kafka.consumer.ConsumerFetcherThread - [ConsumerFetcherThread-group1_square-1a7ac0.local-1386869343370-dc19c7dc-0-1946108683], Error due to
      kafka.common.KafkaException: error processing data for partition [test-bad,0] offset 0
      at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:137)
      at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:109)
      at scala.collection.immutable.Map$Map1.foreach(Map.scala:105)
      at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply$mcV$sp(AbstractFetcherThread.scala:109)
      at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(AbstractFetcherThread.scala:109)
      at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1.apply(AbstractFetcherThread.scala:109)
      at kafka.utils.Utils$.inLock(Utils.scala:565)
      at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:108)
      at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86)
      at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51)
      Caused by: java.lang.NullPointerException
      at kafka.consumer.PartitionTopicInfo.enqueue(PartitionTopicInfo.scala:60)
      at kafka.consumer.ConsumerFetcherThread.processPartitionData(ConsumerFetcherThread.scala:49)
      at kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$1$$anonfun$apply$mcV$sp$2.apply(AbstractFetcherThread.scala:128)
      ... 9 more


        1. apply-patch-1180-to-0.8.1.patch
          2 kB
          Jason Rosenberg
        2. KAFKA-1180_2013-12-22_01:24:57.patch
          2 kB
          Joe Stein
        3. KAFKA-1180_2014-02-09_20:21:49.patch
          4 kB
          Joe Stein
        4. KAFKA-1180_2014-02-13_14:50:40.patch
          9 kB
          Joe Stein
        5. KAFKA-1180_2014-02-13_15:13:17.patch
          5 kB
          Joe Stein
        6. KAFKA-1180_2014-02-13_15:21:51.patch
          4 kB
          Joe Stein
        7. KAFKA-1180_2014-02-13_15:23:28.patch
          4 kB
          Joe Stein
        8. KAFKA-1180.patch
          2 kB
          Joe Stein
        9. KAFKA-1180.patch
          2 kB
          Joe Stein



            • Assignee:
              joestein Joe Stein
              jbrosenberg@gmail.com Jason Rosenberg
            • Votes:
              0 Vote for this issue
              6 Start watching this issue


              • Created: