Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-3978

Cannot truncate to a negative offset (-1) exception at broker startup

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.10.0.0
    • Fix Version/s: 1.1.0
    • Component/s: None
    • Environment:
      3.13.0-87-generic

      Description

      During broker startup sequence the broker server.log has this exception. Problem persists after multiple restarts and also on another broker in the cluster.

      INFO [Socket Server on Broker 1002], Started 1 acceptor threads (kafka.network.SocketServer)
      INFO [Socket Server on Broker 1002], Started 1 acceptor threads (kafka.network.SocketServer)
      INFO [ExpirationReaper-1002], Starting  (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
      INFO [ExpirationReaper-1002], Starting  (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
      INFO [ExpirationReaper-1002], Starting  (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
      INFO [ExpirationReaper-1002], Starting  (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
      INFO [ExpirationReaper-1002], Starting  (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
      INFO [ExpirationReaper-1002], Starting  (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
      INFO [ExpirationReaper-1002], Starting  (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
      INFO [ExpirationReaper-1002], Starting  (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
      INFO [GroupCoordinator 1002]: Starting up. (kafka.coordinator.GroupCoordinator)
      INFO [GroupCoordinator 1002]: Starting up. (kafka.coordinator.GroupCoordinator)
      INFO [GroupCoordinator 1002]: Startup complete. (kafka.coordinator.GroupCoordinator)
      INFO [GroupCoordinator 1002]: Startup complete. (kafka.coordinator.GroupCoordinator)
      INFO [Group Metadata Manager on Broker 1002]: Removed 0 expired offsets in 9 milliseconds. (kafka.coordinator.GroupMetadataManager)
      INFO [Group Metadata Manager on Broker 1002]: Removed 0 expired offsets in 9 milliseconds. (kafka.coordinator.GroupMetadataManager)
      INFO [ThrottledRequestReaper-Produce], Starting  (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
      INFO [ThrottledRequestReaper-Produce], Starting  (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
      INFO [ThrottledRequestReaper-Fetch], Starting  (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
      INFO [ThrottledRequestReaper-Fetch], Starting  (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
      INFO Will not load MX4J, mx4j-tools.jar is not in the classpath (kafka.utils.Mx4jLoader$)
      INFO Will not load MX4J, mx4j-tools.jar is not in the classpath (kafka.utils.Mx4jLoader$)
      INFO Creating /brokers/ids/1002 (is it secure? false) (kafka.utils.ZKCheckedEphemeral)
      INFO Creating /brokers/ids/1002 (is it secure? false) (kafka.utils.ZKCheckedEphemeral)
      INFO Result of znode creation is: OK (kafka.utils.ZKCheckedEphemeral)
      INFO Result of znode creation is: OK (kafka.utils.ZKCheckedEphemeral)
      INFO Registered broker 1002 at path /brokers/ids/1002 with addresses: PLAINTEXT -> EndPoint(172.16.2.22,9092,PLAINTEXT) (kafka.utils.ZkUtils)
      INFO Registered broker 1002 at path /brokers/ids/1002 with addresses: PLAINTEXT -> EndPoint(172.16.2.22,9092,PLAINTEXT) (kafka.utils.ZkUtils)
      INFO Kafka version : 0.10.0.0 (org.apache.kafka.common.utils.AppInfoParser)
      INFO Kafka commitId : b8642491e78c5a13 (org.apache.kafka.common.utils.AppInfoParser)
      INFO [Kafka Server 1002], started (kafka.server.KafkaServer)
      INFO [Kafka Server 1002], started (kafka.server.KafkaServer)
      Error when handling request {controller_id=1004,controller_epoch=1,partition_states=[..REALLY LONG OUTPUT SNIPPED AWAY..], live_leaders=[{id=1004,host=172.16.6.187,port=9092},{id=1003,host=172.16.2.21,port=9092}]} (kafka.server.KafkaApis)
      ERROR java.lang.IllegalArgumentException: Cannot truncate to a negative offset (-1).
              at kafka.log.Log.truncateTo(Log.scala:731)
              at kafka.log.LogManager$$anonfun$truncateTo$2.apply(LogManager.scala:288)
              at kafka.log.LogManager$$anonfun$truncateTo$2.apply(LogManager.scala:280)
              at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:733)
              at scala.collection.immutable.HashMap$HashMap1.foreach(HashMap.scala:221)
              at scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:428)
              at scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:428)
              at scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:428)
              at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:732)
              at kafka.log.LogManager.truncateTo(LogManager.scala:280)
              at kafka.server.ReplicaManager.makeFollowers(ReplicaManager.scala:802)
              at kafka.server.ReplicaManager.becomeLeaderOrFollower(ReplicaManager.scala:648)
              at kafka.server.KafkaApis.handleLeaderAndIsrRequest(KafkaApis.scala:144)
              at kafka.server.KafkaApis.handle(KafkaApis.scala:80)
              at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:60)
              at java.lang.Thread.run(Thread.java:745)
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                lindong Dong Lin
                Reporter:
                garo5 Juho Mäkinen
              • Votes:
                1 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: