Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1875

MiniDFSCluster hard-codes dfs.datanode.address to localhost


    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.23.0
    • Component/s: test
    • Labels:


      When creating RPC addresses that represent the communication sockets for each simulated DataNode, the MiniDFSCluster class hard-codes the address of the dfs.datanode.address port to be ""

      The DataNodeCluster test tool uses the MiniDFSCluster class to create a selected number of simulated datanodes on a single host. In the DataNodeCluster setup, the NameNode is not simulated but is started as a separate daemon.

      The problem is that if the write requrests into the simulated datanodes are originated on a host that is not the same host running the simulated datanodes, the connections are refused. This is because the RPC sockets that are started by MiniDFSCluster are for "localhost" ( and are not accessible from outside that same machine.

      It is proposed that the MiniDFSCluster.setupDatanodeAddress() method be overloaded in order to accommodate an environment where the NameNode is on one host, the client is on another host, and the simulated DataNodes are on yet another host (or even multiple hosts simulating multiple DataNodes each).

      The overloaded API would add a parameter that would be used as the basis for creating the RPS sockets. By default, it would remain

      1. HDFS-1875.patch
        9 kB
        Eric Payne
      2. HDFS-1875.patch
        11 kB
        Eric Payne


        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Matt Foley made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Eric Payne made changes -
        Attachment HDFS-1875.patch [ 12481423 ]
        Eric Payne made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Eric Payne made changes -
        Field Original Value New Value
        Attachment HDFS-1875.patch [ 12479677 ]
        Eric Payne created issue -


          • Assignee:
            Eric Payne
            Eric Payne
          • Votes:
            0 Vote for this issue
            6 Start watching this issue


            • Created: