Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-2427

Error writing to highwatermark file

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Critical
    • Resolution: Unresolved
    • 0.8.2.1
    • None
    • replication
    • None
    • Ubuntu 14.04

    Description

      Periodically one instance of the kafka broker crashes (process exits) with the following error. Note: The persistence of files makes use of NFS mount

      [2015-08-12 08:42:12,480] FATAL [Replica Manager on Broker 1]: Error writing to highwatermark file: (kafka.server.ReplicaManager)
      java.io.IOException: File rename from /nfs/data/kafka1-logs/replication-offset-checkpoint.tmp to /nfs/data/kafka1-logs/replication-offset-checkpoint failed.
      at kafka.server.OffsetCheckpoint.write(OffsetCheckpoint.scala:66)
      at kafka.server.ReplicaManager$$anonfun$checkpointHighWatermarks$2.apply(ReplicaManager.scala:596)
      at kafka.server.ReplicaManager$$anonfun$checkpointHighWatermarks$2.apply(ReplicaManager.scala:593)
      at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:772)
      at scala.collection.immutable.Map$Map1.foreach(Map.scala:109)
      at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:771)
      at kafka.server.ReplicaManager.checkpointHighWatermarks(ReplicaManager.scala:593)
      at kafka.server.ReplicaManager$$anonfun$1.apply$mcV$sp(ReplicaManager.scala:99)
      at kafka.utils.KafkaScheduler$$anonfun$1.apply$mcV$sp(KafkaScheduler.scala:99)
      at kafka.utils.Utils$$anon$1.run(Utils.scala:54)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:745)
      (END)packet_write_wait: Connection to 10.23.2.110: Broken pipe

      Attachments

        Activity

          People

            nehanarkhede Neha Narkhede
            hseshadr Harish Seshadri
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: