Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9444

Add utility to find set of available ephemeral ports to ServerSocketUtil

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 2.7.4, 3.0.0-alpha2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Unit tests using MiniDFSCluster with namanode-ha enabled need set of port numbers in advance. Because namenodes talk to each other, we can not set ipc port to 0 in configuration to make namenodes decide port number on its own. ServerSocketUtil should provide utility to find set of available ephemeral port numbers for this.

      For example, TestEditLogTailer could fail due to java.net.BindException: Address already in use.

      https://builds.apache.org/job/Hadoop-Hdfs-trunk/2556/testReport/

      java.net.BindException: Problem binding to [localhost:42477] java.net.BindException: Address already in use; For more details see:  http://wiki.apache.org/hadoop/BindException
      	at sun.nio.ch.Net.bind0(Native Method)
      	at sun.nio.ch.Net.bind(Net.java:444)
      	at sun.nio.ch.Net.bind(Net.java:436)
      	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214)
      	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
      	at org.apache.hadoop.ipc.Server.bind(Server.java:469)
      	at org.apache.hadoop.ipc.Server$Listener.<init>(Server.java:695)
      	at org.apache.hadoop.ipc.Server.<init>(Server.java:2464)
      	at org.apache.hadoop.ipc.RPC$Server.<init>(RPC.java:945)
      	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server.<init>(ProtobufRpcEngine.java:535)
      	at org.apache.hadoop.ipc.ProtobufRpcEngine.getServer(ProtobufRpcEngine.java:510)
      	at org.apache.hadoop.ipc.RPC$Builder.build(RPC.java:787)
      	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.<init>(NameNodeRpcServer.java:390)
      	at org.apache.hadoop.hdfs.server.namenode.NameNode.createRpcServer(NameNode.java:742)
      	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:680)
      	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:883)
      	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:862)
      	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1564)
      	at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNode(MiniDFSCluster.java:1247)
      	at org.apache.hadoop.hdfs.MiniDFSCluster.configureNameService(MiniDFSCluster.java:1016)
      	at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:891)
      	at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:823)
      	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:482)
      	at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:441)
      	at org.apache.hadoop.hdfs.server.namenode.ha.TestEditLogTailer.testStandbyTriggersLogRolls(TestEditLogTailer.java:139)
      	at org.apache.hadoop.hdfs.server.namenode.ha.TestEditLogTailer.testNN1TriggersLogRolls(TestEditLogTailer.java:114)
      

        Attachments

        1. HDFS-9444.001.patch
          3 kB
          Masatake Iwasaki
        2. HDFS-9444.002.patch
          4 kB
          Masatake Iwasaki
        3. HDFS-9444.003.patch
          4 kB
          Masatake Iwasaki
        4. HDFS-9444.004.patch
          7 kB
          Masatake Iwasaki
        5. HDFS-9444.005.patch
          7 kB
          Xiao Chen
        6. HDFS-9444.006.patch
          7 kB
          Xiao Chen
        7. HDFS-9444-branch-2.006.patch
          6 kB
          Masatake Iwasaki
        8. HDFS-9444-branch-2.007.patch
          7 kB
          Masatake Iwasaki

          Issue Links

            Activity

              People

              • Assignee:
                iwasakims Masatake Iwasaki
                Reporter:
                brahmareddy Brahma Reddy Battula
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: