[KAFKA-3453] Transient test failures due to MiniKDC port allocation strategy

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.9.0.1
• Fix Version/s: 0.10.0.0
• Assignee: Ismael Juma
• Reporter: Ewen Cheslack-Postava
• Reviewer: Gwen Shapira
• Created: 23/Mar/16 20:05
• Updated: 31/Mar/16 09:12
• Resolved: 31/Mar/16 09:11

Description

A number of tests, especially our consumer tests, fail transiently because MiniKDC allocates ports by creating a socket, getting its port, then closing it. As previously addressed in our own code, this causes problems because that port can be reallocated before the process has a chance to bind a new socket – whether due to another test running in parallel or another process simply binding the port first. This results in errors like this in the tests:

java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:444)
at sun.nio.ch.Net.bind(Net.java:436)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.apache.mina.transport.socket.nio.NioSocketAcceptor.open(NioSocketAcceptor.java:198)
at org.apache.mina.transport.socket.nio.NioSocketAcceptor.open(NioSocketAcceptor.java:51)
at org.apache.mina.core.polling.AbstractPollingIoAcceptor.registerHandles(AbstractPollingIoAcceptor.java:547)
at org.apache.mina.core.polling.AbstractPollingIoAcceptor.access$400(AbstractPollingIoAcceptor.java:68)
at org.apache.mina.core.polling.AbstractPollingIoAcceptor$Acceptor.run(AbstractPollingIoAcceptor.java:422)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:64)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

This is an ongoing issue that Confluent sees in its Jenkins builds, which is the reason for this ticket. The real issue is actually in MiniKDC (we pass in "0" for the port, but then it uses this other port allocation strategy), but we either need to a) figure out a workaround or b) get a fix in upstream and then update to a newer MiniKDC version.