[KAFKA-4271] The console consumer fails on Windows with new consumer is used

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.10.0.1
• Fix Version/s: 0.10.1.1, 0.10.2.0
• Assignee: Vahid Hashemian
• Reporter: Vahid Hashemian
• Created: 07/Oct/16 18:26
• Updated: 30/Nov/16 20:20
• Resolved: 30/Nov/16 20:20

Description

When I try to consume message using the new consumer (Quickstart Step 5) I get an exception on the broker side. The old consumer works fine.

java.io.IOException: Map failed
        at sun.nio.ch.FileChannelImpl.map(Unknown Source)
        at kafka.log.AbstractIndex.<init>(AbstractIndex.scala:61)
        at kafka.log.OffsetIndex.<init>(OffsetIndex.scala:51)
        at kafka.log.LogSegment.<init>(LogSegment.scala:67)
        at kafka.log.Log.loadSegments(Log.scala:255)
        at kafka.log.Log.<init>(Log.scala:108)
        at kafka.log.LogManager.createLog(LogManager.scala:362)
        at kafka.cluster.Partition.getOrCreateReplica(Partition.scala:94)
        at kafka.cluster.Partition$$anonfun$4$$anonfun$apply$2.apply(Partition.scala:174)
        at kafka.cluster.Partition$$anonfun$4$$anonfun$apply$2.apply(Partition.scala:174)
        at scala.collection.mutable.HashSet.foreach(HashSet.scala:79)
        at kafka.cluster.Partition$$anonfun$4.apply(Partition.scala:174)
        at kafka.cluster.Partition$$anonfun$4.apply(Partition.scala:168)
        at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:234)
        at kafka.utils.CoreUtils$.inWriteLock(CoreUtils.scala:242)
        at kafka.cluster.Partition.makeLeader(Partition.scala:168)
        at kafka.server.ReplicaManager$$anonfun$makeLeaders$4.apply(ReplicaManager.scala:740)
        at kafka.server.ReplicaManager$$anonfun$makeLeaders$4.apply(ReplicaManager.scala:739)
        at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98)
        at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98)
        at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:226)
        at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39)
        at scala.collection.mutable.HashMap.foreach(HashMap.scala:98)
        at kafka.server.ReplicaManager.makeLeaders(ReplicaManager.scala:739)
        at kafka.server.ReplicaManager.becomeLeaderOrFollower(ReplicaManager.scala:685)
        at kafka.server.KafkaApis.handleLeaderAndIsrRequest(KafkaApis.scala:148)
        at kafka.server.KafkaApis.handle(KafkaApis.scala:82)
        at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:60)
        at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.OutOfMemoryError: Map failed
        at sun.nio.ch.FileChannelImpl.map0(Native Method)
        ... 29 more

This issue seems to break the broker and I have to clear out the logs so I can bring the broker back up again.

Update: This issue seems to occur on 32-bit Windows only. I tried this on a Windows 32-bit VM. On a 64-bit machine I did not get the error.

Issue Links

• links to: GitHub Pull Request #2189