[HADOOP-15311] HttpServer2 needs a way to configure the acceptor/selector count

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.1.0, 3.0.3
• Component/s: common
• Hadoop Flags: Reviewed
• Assignee: Erik Krogen
• Reporter: Erik Krogen
• Created: 13/Mar/18 16:30
• Updated: 05/Apr/18 19:36
• Resolved: 13/Mar/18 21:22

Description

HttpServer2 starts up with some number of acceptors and selectors, but only allows for the automatic configuration of these based off of the number of available cores:

org.eclipse.jetty.server.ServerConnector

selectors > 0 ? selectors : Math.max(1, Math.min(4, Runtime.getRuntime().availableProcessors() / 2)))

org.eclipse.jetty.server.AbstractConnector

    if (acceptors < 0) {
      acceptors = Math.max(1, Math.min(4, cores / 8));
    }

A thread pool is started of size, at minimum, acceptors + selectors + 1, so in addition to allowing for a higher tuning value under heavily loaded environments, adding configurability for this enables tuning these values down in resource constrained environments such as a MiniDFSCluster.

Issue Links

• is depended upon by: HDFS-13265 MiniDFSCluster should set reasonable defaults to reduce resource consumption