Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-877

Still getting kafka.common.NotLeaderForPartitionException

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Cannot Reproduce
    • Affects Version/s: 0.8.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Environment:
      DEV

      Description

      Using the below trunk and i still see error happening.Please let us know if this can be fixed.

      https://github.com/apache/kafka.git

      [2013-04-25 16:47:08,924] WARN [console-consumer-24019_MERD7-21964-1366930009136-8b7f9eb7-leader-finder-thread], Failed to add fetcher for [mytopic,0] to broker id:0,host:MERD7-21964.echostar.com,port:9092 (kafka.consumer.ConsumerFetcherManager$$anon$1)
      kafka.common.NotLeaderForPartitionException
      at sun.reflect.GeneratedConstructorAccessor1.newInstance(Unknown Source)
      at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
      at java.lang.Class.newInstance0(Class.java:372)
      at java.lang.Class.newInstance(Class.java:325)
      at kafka.common.ErrorMapping$.exceptionFor(ErrorMapping.scala:72)
      at kafka.consumer.SimpleConsumer.earliestOrLatestOffset(SimpleConsumer.scala:163)
      at kafka.consumer.ConsumerFetcherThread.handleOffsetOutOfRange(ConsumerFetcherThread.scala:61)
      at kafka.server.AbstractFetcherThread.addPartition(AbstractFetcherThread.scala:167)
      at kafka.server.AbstractFetcherManager.addFetcher(AbstractFetcherManager.scala:48)
      at kafka.consumer.ConsumerFetcherManager$$anon$1$$anonfun$doWork$3.apply(ConsumerFetcherManager.scala:79)
      at kafka.consumer.ConsumerFetcherManager$$anon$1$$anonfun$doWork$3.apply(ConsumerFetcherManager.scala:75)
      at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:95)
      at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:95)
      at scala.collection.Iterator$class.foreach(Iterator.scala:772)
      at scala.collection.mutable.HashTable$$anon$1.foreach(HashTable.scala:157)
      at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:190)
      at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:45)
      at scala.collection.mutable.HashMap.foreach(HashMap.scala:95)
      at kafka.consumer.ConsumerFetcherManager$$anon$1.doWork(ConsumerFetcherManager.scala:75)
      at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51)

      We are evaluating Kafka for our new messaging system and we had tough time running in windows.

      We somehow managed to run 0.8 using cygwin but when we run the console producer/consumer,we are not getting messages from consumer.

      Please help us to fix this issue,this might not be related but its keeping on throwing this error on consumer side.

      1. KAFKA-816.jpg
        62 kB
        BalajiSeshadri

        Issue Links

          Activity

          Hide
          balaji.seshadri@dish.com BalajiSeshadri added a comment -

          I'm using 0.8 branch and it still occurs

          Show
          balaji.seshadri@dish.com BalajiSeshadri added a comment - I'm using 0.8 branch and it still occurs
          Hide
          nehanarkhede Neha Narkhede added a comment -

          We have fixed a number of bugs related to this in 08 recently. Do you mind trying it out again ?

          Also, would you mind describing in a separate JIRA what problems you had while trying to run Kafka on Windows?

          Show
          nehanarkhede Neha Narkhede added a comment - We have fixed a number of bugs related to this in 08 recently. Do you mind trying it out again ? Also, would you mind describing in a separate JIRA what problems you had while trying to run Kafka on Windows?
          Hide
          jkreps Jay Kreps added a comment -

          Is this still happening? If not I want to mark it closed...

          Show
          jkreps Jay Kreps added a comment - Is this still happening? If not I want to mark it closed...
          Hide
          balaji.seshadri@dish.com BalajiSeshadri added a comment -

          Please close it,we will reopen it if it happens again.

          Show
          balaji.seshadri@dish.com BalajiSeshadri added a comment - Please close it,we will reopen it if it happens again.
          Hide
          jkreps Jay Kreps added a comment -

          Will reopen if it reoccurs.

          Show
          jkreps Jay Kreps added a comment - Will reopen if it reoccurs.
          Hide
          rthille Robert P. Thille added a comment -

          We see this in Kafka 0.8.2.1 when our systems get overloaded and either ZooKeeper gets stalled trying to fsync, or Kafka gets starved and loses its connection to ZK. It takes a restart of the brokers to get them properly in sync thereafter.

          Show
          rthille Robert P. Thille added a comment - We see this in Kafka 0.8.2.1 when our systems get overloaded and either ZooKeeper gets stalled trying to fsync, or Kafka gets starved and loses its connection to ZK. It takes a restart of the brokers to get them properly in sync thereafter.
          Hide
          rthille Robert P. Thille added a comment - - edited

          I have logs from a 3-node cluster which got into a bad state after ZK expired:

          [2017-05-09 18:33:42,897] INFO [SessionExpirationListener on 2], ZK expired; shut down all controller components and try to re-elect (kafka.controller.KafkaController$SessionExpirationListener)
          [2017-05-17 05:06:13,469] INFO [SessionExpirationListener on 2], ZK expired; shut down all controller components and try to re-elect (kafka.controller.KafkaController$SessionExpirationListener)
          [2017-05-17 16:33:43,349] INFO [SessionExpirationListener on 2], ZK expired; shut down all controller components and try to re-elect (kafka.controller.KafkaController$SessionExpirationListener)
          [2017-05-17 16:33:44,059] INFO [Controller 2]: Broker 2 starting become controller state transition (kafka.controller.KafkaController)
          

          The first two ZK session expirations were handled fine. The 3rd blew everything up.

          Show
          rthille Robert P. Thille added a comment - - edited I have logs from a 3-node cluster which got into a bad state after ZK expired: [2017-05-09 18:33:42,897] INFO [SessionExpirationListener on 2], ZK expired; shut down all controller components and try to re-elect (kafka.controller.KafkaController$SessionExpirationListener) [2017-05-17 05:06:13,469] INFO [SessionExpirationListener on 2], ZK expired; shut down all controller components and try to re-elect (kafka.controller.KafkaController$SessionExpirationListener) [2017-05-17 16:33:43,349] INFO [SessionExpirationListener on 2], ZK expired; shut down all controller components and try to re-elect (kafka.controller.KafkaController$SessionExpirationListener) [2017-05-17 16:33:44,059] INFO [Controller 2]: Broker 2 starting become controller state transition (kafka.controller.KafkaController) The first two ZK session expirations were handled fine. The 3rd blew everything up.

            People

            • Assignee:
              Unassigned
              Reporter:
              balaji.seshadri@dish.com BalajiSeshadri
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development