Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
0.22.0
-
None
-
None
-
Ubuntu Linux, kernel 2.6.24-25-server, x86_64 architecture. Java 1.6.0., OpenJDK. Server has 4 physical Ethernet cards, configured as follows: eth0 is assigned an IP address external-ip. eth1-eth3 are bonded as bond0, which is assigned an IP address internal-ip. This server has two dns entries: external-dns which resolves to external-ip and internal-dns which resolves to internal-ip.
Ubuntu Linux, kernel 2.6.24-25-server, x86_64 architecture. Java 1.6.0., OpenJDK. Server has 4 physical Ethernet cards, configured as follows: eth0 is assigned an IP address external-ip . eth1 - eth3 are bonded as bond0 , which is assigned an IP address internal-ip . This server has two dns entries: external-dns which resolves to external-ip and internal-dns which resolves to internal-ip .
Description
High-level:
In my configuration files, I specifiy:
<property> <name>fs.default.name</name> <value>hdfs://{{internal-dns}}:8020</value> </property>
The name node server binds to address external-ip:8020, while all the data nodes try to connect to internal-ip:8020, and nothing works.
Low-level: I've traced this down as far as org.apache.hadoop.ipc.Server, private class Listener.
/** Listens on the socket. Creates jobs for the handler threads*/ private class Listener extends Thread { private ServerSocketChannel acceptChannel = null; //the accept channel private Selector selector = null; //the selector that we use for the server private InetSocketAddress address; //the address we bind at private Random rand = new Random(); private long lastCleanupRunTime = 0; //the last time when a cleanup connec- //-tion (for idle connections) ran private long cleanupInterval = 10000; //the minimum interval between //two cleanup runs private int backlogLength = conf.getInt("ipc.server.listen.queue.size", 128); public Listener() throws IOException { address = new InetSocketAddress(bindAddress, port); LOG.info("in Server/Listener. bindAddress=" + bindAddress + ", address="+address); //Added by EA for testing // Create a new server socket and set to non blocking mode acceptChannel = ServerSocketChannel.open(); acceptChannel.configureBlocking(false); // Bind the server socket to the local host and port bind(acceptChannel.socket(), address, backlogLength); port = acceptChannel.socket().getLocalPort(); //Could be an ephemeral port
The problem seems to be in the line
address = new InetSocketAddress(bindAddress, port);
. bindAddress contains the DNS name of the internal interface, but address comes out with the IP address of the external interface.