Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15311

HttpServer2 needs a way to configure the acceptor/selector count

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.1.0, 3.0.3
    • Component/s: common
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      HttpServer2 starts up with some number of acceptors and selectors, but only allows for the automatic configuration of these based off of the number of available cores:

      org.eclipse.jetty.server.ServerConnector
      selectors > 0 ? selectors : Math.max(1, Math.min(4, Runtime.getRuntime().availableProcessors() / 2)))
      
      org.eclipse.jetty.server.AbstractConnector
          if (acceptors < 0) {
            acceptors = Math.max(1, Math.min(4, cores / 8));
          }
      

      A thread pool is started of size, at minimum, acceptors + selectors + 1, so in addition to allowing for a higher tuning value under heavily loaded environments, adding configurability for this enables tuning these values down in resource constrained environments such as a MiniDFSCluster.

        Attachments

        1. HADOOP-15311.002.patch
          4 kB
          Chris Douglas
        2. HADOOP-15311.001.patch
          4 kB
          Erik Krogen
        3. HADOOP-15311.000.patch
          4 kB
          Erik Krogen

          Issue Links

            Activity

              People

              • Assignee:
                xkrogen Erik Krogen
                Reporter:
                xkrogen Erik Krogen
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: