[KUDU-776] Improve handling of unresolvable master hostnames

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: M5
• Component/s: client
• Assignee: Jean-Daniel Cryans
• Reporter: Jean-Daniel Cryans
• Created: 15/May/15 17:31
• Updated: 26/Feb/16 12:08
• Resolved: 18/May/15 18:25

Description

I tried starting a MR job against a malformed hostname (locahost), got this and it just kept on spewing:

2015-05-15 15:28:18,549 (main) [INFO - org.apache.hadoop.metrics.jvm.JvmMetrics.init(JvmMetrics.java:76)] Initializing JVM Metrics with processName=JobTracker, sessionId=
2015-05-15 15:28:19,477 (main) [ERROR - org.kududb.client.TabletClient.exceptionCaught(TabletClient.java:644)] Unexpected exception from downstream on [id: 0xe1ed8321]
java.nio.channels.UnresolvedAddressException
	at sun.nio.ch.Net.checkAddress(Net.java:127)
	at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:644)
	at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:108)
	at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:70)
	at org.kududb.client.AsyncKuduClient$TabletClientPipeline.sendDownstream(AsyncKuduClient.java:1458)
	at org.jboss.netty.channel.Channels.connect(Channels.java:634)
	at org.jboss.netty.channel.AbstractChannel.connect(AbstractChannel.java:207)
	at org.kududb.client.AsyncKuduClient.newClient(AsyncKuduClient.java:1187)
	at org.kududb.client.AsyncKuduClient.newMasterClient(AsyncKuduClient.java:1160)
	at org.kududb.client.AsyncKuduClient.getMasterTableLocationsPB(AsyncKuduClient.java:873)
	at org.kududb.client.AsyncKuduClient.locateTablet(AsyncKuduClient.java:845)
	at org.kududb.client.AsyncKuduClient.sendRpcToTablet(AsyncKuduClient.java:555)
	at org.kududb.client.AsyncKuduClient.getTableSchema(AsyncKuduClient.java:345)
	at org.kududb.client.AsyncKuduClient.openTable(AsyncKuduClient.java:397)
	at org.kududb.client.KuduClient.openTable(KuduClient.java:189)
	at org.kududb.mapreduce.KuduTableInputFormat.setConf(KuduTableInputFormat.java:225)
	at ...
2015-05-15 15:28:19,481 (main) [ERROR - org.kududb.client.AsyncKuduClient.getIP(AsyncKuduClient.java:1541)] Failed to resolve the IP of `localhot' in 55597ns
2015-05-15 15:28:19,481 (main) [ERROR - org.kududb.client.AsyncKuduClient$TabletClientPipeline.handleDisconnect(AsyncKuduClient.java:1505)] Uncaught exception when handling a disconnection of [id: 0xe1ed8321]
java.lang.IllegalArgumentException: hostname can't be null
	at java.net.InetSocketAddress.checkHost(InetSocketAddress.java:149)
	at java.net.InetSocketAddress.<init>(InetSocketAddress.java:216)
	at org.kududb.client.AsyncKuduClient.slowSearchClientIP(AsyncKuduClient.java:1371)
	at org.kududb.client.AsyncKuduClient.access$1500(AsyncKuduClient.java:112)
	at org.kududb.client.AsyncKuduClient$TabletClientPipeline.handleDisconnect(AsyncKuduClient.java:1496)
	at org.kududb.client.AsyncKuduClient$TabletClientPipeline.sendDownstream(AsyncKuduClient.java:1456)
	at org.jboss.netty.channel.Channels.close(Channels.java:812)
	at org.kududb.client.TabletClient.exceptionCaught(TabletClient.java:647)
	at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112)
	at org.kududb.client.TabletClient.handleUpstream(TabletClient.java:568)
	at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
	at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
	at org.kududb.client.AsyncKuduClient$TabletClientPipeline.sendUpstream(AsyncKuduClient.java:1466)
	at org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:525)
	...
2015-05-15 15:28:19,488 (main) [WARN - org.kududb.client.GetMasterRegistrationReceived$GetMasterRegistrationErrCB.call(GetMasterRegistrationReceived.java:171)] Error receiving a response from: localhot:61000
org.kududb.client.ConnectionResetException
	at org.kududb.client.TabletClient.sendRpc(TabletClient.java:175)
	at org.kududb.client.AsyncKuduClient.getMasterRegistration(AsyncKuduClient.java:1148)
	at org.kududb.client.AsyncKuduClient.getMasterTableLocationsPB(AsyncKuduClient.java:874)
	at org.kududb.client.AsyncKuduClient.locateTablet(AsyncKuduClient.java:845)
	at org.kududb.client.AsyncKuduClient.sendRpcToTablet(AsyncKuduClient.java:555)
	at org.kududb.client.AsyncKuduClient.getTableSchema(AsyncKuduClient.java:345)
	at org.kududb.client.AsyncKuduClient.openTable(AsyncKuduClient.java:397)
	at org.kududb.client.KuduClient.openTable(KuduClient.java:189)
	at org.kududb.mapreduce.KuduTableInputFormat.setConf(KuduTableInputFormat.java:225)
	at ...