[AMBARI-13290] Set kafka userid ulimit open files to 32k for kafka broker through Ambari

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.2.0
• Assignee: Andrew Onischuk
• Reporter: Andrew Onischuk
• Created: 01/Oct/15 16:38
• Updated: 02/Oct/15 15:31
• Resolved: 02/Oct/15 11:41

Description

PROBLEM: Currently Ambari does not set any ulimit for the kafka user during
install.

[2015-09-22 07:01:01,380] FATAL [Replica Manager on Broker 2]: Error writing to highwatermark file: (kafka.server.ReplicaManager)
java.io.FileNotFoundException: /mnt/data2/kafka-logs/replication-offset-checkpoint.tmp (Too many open files)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
.
.
[2015-09-22 07:01:01,380] ERROR Error in acceptor (kafka.network.Acceptor)
java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:241)
at kafka.network.Acceptor.accept(SocketServer.scala:200)
at kafka.network.Acceptor.run(SocketServer.scala:154)
at java.lang.Thread.run(Thread.java:745)

The open files limit in ulimits for kafka userid was set to 1024. We increased
to 32000 and got over the error.

Issue Links

• links to: ReviewBoard