Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Duplicate
-
0.8.0
-
None
-
None
-
Windows 7 64bit
Description
When a large number of messages are sent per second to a broker on Windows a memory mapping exception occurs and kills Kafka. The exception follows :
kafka.common.KafkaStorageException: I/O exception in append to log 'test-0' at kafka.log.Log.append(Log.scala:349) at kafka.cluster.Partition.appendMessagesToLeader(Partition.scala:340) at kafka.server.KafkaApis$$anonfun$appendToLocalLog$2.apply(KafkaApis.scala:236) at kafka.server.KafkaApis$$anonfun$appendToLocalLog$2.apply(KafkaApis.scala:228) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206) at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:80) at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:80) at scala.collection.Iterator$class.foreach(Iterator.scala:631) at scala.collection.mutable.HashTable$$anon$1.foreach(HashTable.scala:161) at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:194) at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39) at scala.collection.mutable.HashMap.foreach(HashMap.scala:80) at scala.collection.TraversableLike$class.map(TraversableLike.scala:206) at scala.collection.mutable.HashMap.map(HashMap.scala:39) at kafka.server.KafkaApis.appendToLocalLog(KafkaApis.scala:228) at kafka.server.KafkaApis.handleProducerRequest(KafkaApis.scala:162) at kafka.server.KafkaApis.handle(KafkaApis.scala:66) at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:42) at java.lang.Thread.run(Thread.java:662) Caused by: java.io.IOException: The requested operation cannot be performed on a file with a user-mapped section open at java.io.RandomAccessFile.setLength(Native Method) at kafka.log.OffsetIndex.liftedTree2$1(OffsetIndex.scala:263) at kafka.log.OffsetIndex.resize(OffsetIndex.scala:262) at kafka.log.OffsetIndex.trimToValidSize(OffsetIndex.scala:247) at kafka.log.Log.rollToOffset(Log.scala:518) at kafka.log.Log.roll(Log.scala:502) at kafka.log.Log.maybeRoll(Log.scala:484) at kafka.log.Log.append(Log.scala:297) ... 19 more
This seems to have been mentioned in the past on the Kafka mail list and is an issue related to http://bugs.sun.com/view_bug.do?bug_id=4724038.
Unfortunately this means that we cannot use Kafka on Windows.
Attachments
Issue Links
- is duplicated by
-
KAFKA-1008 Unmap before resizing
- Resolved