Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-880

NoLeaderPartitionSet should be cleared before leader finder thread is started up

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: 0.8.0
    • Component/s: None
    • Labels:
      None

      Description

      This was a recent regression.

      This could prevent the consumer from progressing because fetchers for the currently owned partitions may not be added (depending on the order that the map iterator yields).

      I think the fix should be simple - just clear the set after stopping the leader finder thread and stopping fetchers.

      [2013-04-25 17:06:38,377] WARN [sometopic-somehost-1366909575615-f801367d-leader-finder-thread]
      , Failed to find leader for Set([sometopic,11], [sometopic,25], [sometopic,24]) (kafka.consumer.ConsumerFetcherManager$Lead
      erFinderThread)
      java.util.NoSuchElementException: key not found: [sometopic,24]
      at scala.collection.MapLike$class.default(MapLike.scala:223)
      at scala.collection.immutable.Map$Map2.default(Map.scala:110)
      at scala.collection.MapLike$class.apply(MapLike.scala:134)
      at scala.collection.immutable.Map$Map2.apply(Map.scala:110)
      at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread$$anonfun$doWork$4.apply(ConsumerFetcherManager.scala:81)
      at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread$$anonfun$doWork$4.apply(ConsumerFetcherManager.scala:79)
      at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:80)
      at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:80)
      at scala.collection.Iterator$class.foreach(Iterator.scala:631)
      at scala.collection.mutable.HashTable$$anon$1.foreach(HashTable.scala:161)
      at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:194)
      at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39)
      at scala.collection.mutable.HashMap.foreach(HashMap.scala:80)
      at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:79)
      at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51)

        Attachments

        1. kafka-880.patch
          0.7 kB
          Jun Rao

          Activity

            People

            • Assignee:
              junrao Jun Rao
              Reporter:
              jjkoshy Joel Jacob Koshy
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: