[ZOOKEEPER-2366] Reconfiguration of client port causes a socket leak

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 3.5.0
• Fix Version/s: 3.5.2, 3.6.0
• Component/s: quorum
• Hadoop Flags: Reviewed
• Assignee: Flavio Paiva Junqueira
• Reporter: Timothy James Ward
• Created: 12/Feb/16 16:59
• Updated: 21/Jul/16 20:18
• Resolved: 23/Jun/16 21:46

Description

The NIOServerCnxnFactory reconfigure method can leak server sockets, and hence make ports unusable until the JVM restarts:

The first line of the method takes a reference to the current ServerSocketChannel and then the next line replaces it. The subsequent interactions with the server socket can fail (for example if the reconfiguration tries to bind to an in-use port). If they fail before the call to oldSS.close() then oldSS is never closed. This holds that port open forever, and prevents the user from rolling back to the previous port!

The code from reconfigure is shown below:

ServerSocketChannel oldSS = ss;
try {
this.ss = ServerSocketChannel.open();
ss.socket().setReuseAddress(true);
LOG.info("binding to port " + addr);
ss.socket().bind(addr);
ss.configureBlocking(false);
acceptThread.setReconfiguring();
oldSS.close();
acceptThread.wakeupSelector();
try

{ acceptThread.join(); }

catch (InterruptedException e)

{ LOG.error("Error joining old acceptThread when reconfiguring client port " + e.getMessage()); }

acceptThread = new AcceptThread(ss, addr, selectorThreads);
acceptThread.start();
} catch(IOException e)

{ LOG.error("Error reconfiguring client port to " + addr + " " + e.getMessage()); }