Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-6610

initial high watermark -1, used for truncation, cause "Cannot truncate to a negative offset"

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 0.11.0.2
    • Fix Version/s: None
    • Component/s: log, replication
    • Labels:
      None

      Description

      Hello,

      I got issue that cause the Kafka broker not in-sync with topics. When I check the log, found one of the topic has "-1" highwatermark, then used as the offset for truncating and later caused exception: 

      3/5/2018 10:32:26 AM[2018-03-05 16:32:26,576] ERROR [ReplicaFetcherThread-0-1]: Error due to (kafka.server.ReplicaFetcherThread)
      3/5/2018 10:32:26 AMjava.lang.IllegalArgumentException: Cannot truncate to a negative offset (-1).
      3/5/2018 10:32:26 AM at kafka.log.Log.truncateTo(Log.scala:1377)
      3/5/2018 10:32:26 AM at kafka.log.LogManager.$anonfun$truncateTo$2(LogManager.scala:330)
      3/5/2018 10:32:26 AM at kafka.log.LogManager.$anonfun$truncateTo$2$adapted(LogManager.scala:321)
      3/5/2018 10:32:26 AM at scala.collection.TraversableLike$WithFilter.$anonfun$foreach$1(TraversableLike.scala:789)
      3/5/2018 10:32:26 AM at scala.collection.mutable.HashMap.$anonfun$foreach$1(HashMap.scala:138)
      3/5/2018 10:32:26 AM at scala.collection.mutable.HashTable.foreachEntry(HashTable.scala:236)
      3/5/2018 10:32:26 AM at scala.collection.mutable.HashTable.foreachEntry$(HashTable.scala:229)
      3/5/2018 10:32:26 AM at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)
      3/5/2018 10:32:26 AM at scala.collection.mutable.HashMap.foreach(HashMap.scala:138)
      3/5/2018 10:32:26 AM at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:788)
      3/5/2018 10:32:26 AM at kafka.log.LogManager.truncateTo(LogManager.scala:321)
      3/5/2018 10:32:26 AM at kafka.server.ReplicaFetcherThread.maybeTruncate(ReplicaFetcherThread.scala:279)
      3/5/2018 10:32:26 AM at kafka.server.AbstractFetcherThread.$anonfun$maybeTruncate$2(AbstractFetcherThread.scala:133)
      3/5/2018 10:32:26 AM at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
      3/5/2018 10:32:26 AM at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:213)
      3/5/2018 10:32:26 AM at kafka.server.AbstractFetcherThread.maybeTruncate(AbstractFetcherThread.scala:130)
      3/5/2018 10:32:26 AM at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:102)
      3/5/2018 10:32:26 AM at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:64)

      I deleted the log directory of that topic but topic partition got re-created after restart, with same error after. I'm wondering how the `highwatermark` get set to -1. I failed to trace the code flow from the `Replica` constructor. Thank you.

       

      Dongyan

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                dongyan.li dongyan li
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: