[HDFS-9444] Add utility to find set of available ephemeral ports to ServerSocketUtil

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.8.0, 2.7.4, 3.0.0-alpha2
• Hadoop Flags: Reviewed
• Assignee: Masatake Iwasaki
• Reporter: Brahma Reddy Battula
• Created: 20/Nov/15 16:05
• Updated: 06/Jan/17 08:06
• Resolved: 28/Sep/16 05:51

Description

Unit tests using MiniDFSCluster with namanode-ha enabled need set of port numbers in advance. Because namenodes talk to each other, we can not set ipc port to 0 in configuration to make namenodes decide port number on its own. ServerSocketUtil should provide utility to find set of available ephemeral port numbers for this.

For example, TestEditLogTailer could fail due to java.net.BindException: Address already in use.

https://builds.apache.org/job/Hadoop-Hdfs-trunk/2556/testReport/

java.net.BindException: Problem binding to [localhost:42477] java.net.BindException: Address already in use; For more details see:  http://wiki.apache.org/hadoop/BindException
	at sun.nio.ch.Net.bind0(Native Method)
	at sun.nio.ch.Net.bind(Net.java:444)
	at sun.nio.ch.Net.bind(Net.java:436)
	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
	at org.apache.hadoop.ipc.Server.bind(Server.java:469)
	at org.apache.hadoop.ipc.Server$Listener.<init>(Server.java:695)
	at org.apache.hadoop.ipc.Server.<init>(Server.java:2464)
	at org.apache.hadoop.ipc.RPC$Server.<init>(RPC.java:945)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server.<init>(ProtobufRpcEngine.java:535)
	at org.apache.hadoop.ipc.ProtobufRpcEngine.getServer(ProtobufRpcEngine.java:510)
	at org.apache.hadoop.ipc.RPC$Builder.build(RPC.java:787)
	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.<init>(NameNodeRpcServer.java:390)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.createRpcServer(NameNode.java:742)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:680)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:883)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:862)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1564)
	at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNode(MiniDFSCluster.java:1247)
	at org.apache.hadoop.hdfs.MiniDFSCluster.configureNameService(MiniDFSCluster.java:1016)
	at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:891)
	at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:823)
	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:482)
	at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:441)
	at org.apache.hadoop.hdfs.server.namenode.ha.TestEditLogTailer.testStandbyTriggersLogRolls(TestEditLogTailer.java:139)
	at org.apache.hadoop.hdfs.server.namenode.ha.TestEditLogTailer.testNN1TriggersLogRolls(TestEditLogTailer.java:114)

Issue Links

• is depended upon by: HDFS-9587 Modify tests that initialize MiniDFSNNTopology with hard-coded ports to use ServerSocketUtil#getPorts