[HADOOP-17119] Jetty upgrade to 9.4.x causes MR app fail with IOException

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.1
• Fix Version/s: 3.2.2, 3.3.1, 3.4.0, 3.2.3
• Component/s: common
• Target Version/s: 3.2.2, 3.3.1, 3.4.0, 3.2.3
• Hadoop Flags: Reviewed
• Assignee: Bilwa S T
• Reporter: Bilwa S T
• Created: 08/Jul/20 17:17
• Updated: 11/Feb/24 12:52
• Resolved: 20/Jul/20 17:01

Description

I think we should catch IOException here instead of BindException in HttpServer2#bindForPortRange

 for(Integer port : portRanges) {
      if (port == startPort) {
        continue;
      }
      Thread.sleep(100);
      listener.setPort(port);
      try {
        bindListener(listener);
        return;
      } catch (BindException ex) {
        // Ignore exception. Move to next port.
        ioException = ex;
      }
    }

Stacktrace:

 HttpServer.start() threw a non Bind IOException | HttpServer2.java:1142
java.io.IOException: Failed to bind to xxxxx/xxx.xx.xx.xx:27101
	at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:346)
	at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:307)
	at org.apache.hadoop.http.HttpServer2.bindListener(HttpServer2.java:1190)
	at org.apache.hadoop.http.HttpServer2.bindForPortRange(HttpServer2.java:1258)
	at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:1282)
	at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:1139)
	at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:451)
	at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:440)
	at org.apache.hadoop.mapreduce.v2.app.client.MRClientService.serviceStart(MRClientService.java:148)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1378)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$7.run(MRAppMaster.java:1998)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1994)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1890)
Caused by: java.net.BindException: Address already in use
	at sun.nio.ch.Net.bind0(Native Method)
	at sun.nio.ch.Net.bind(Net.java:433)
	at sun.nio.ch.Net.bind(Net.java:425)
	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:220)
	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:85)
	at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:342)
	... 17 more

Issue Links

• is broken by: HADOOP-16152 Upgrade Eclipse Jetty version to 9.4.x
• relates to: HBASE-24197 TestHttpServer.testBindAddress failure with latest jetty