Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15311

HttpServer2 needs a way to configure the acceptor/selector count

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.1.0, 3.0.3
    • common
    • None
    • Reviewed

    Description

      HttpServer2 starts up with some number of acceptors and selectors, but only allows for the automatic configuration of these based off of the number of available cores:

      org.eclipse.jetty.server.ServerConnector
      selectors > 0 ? selectors : Math.max(1, Math.min(4, Runtime.getRuntime().availableProcessors() / 2)))
      
      org.eclipse.jetty.server.AbstractConnector
          if (acceptors < 0) {
            acceptors = Math.max(1, Math.min(4, cores / 8));
          }
      

      A thread pool is started of size, at minimum, acceptors + selectors + 1, so in addition to allowing for a higher tuning value under heavily loaded environments, adding configurability for this enables tuning these values down in resource constrained environments such as a MiniDFSCluster.

      Attachments

        1. HADOOP-15311.000.patch
          4 kB
          Erik Krogen
        2. HADOOP-15311.001.patch
          4 kB
          Erik Krogen
        3. HADOOP-15311.002.patch
          4 kB
          Christopher Douglas

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            xkrogen Erik Krogen
            xkrogen Erik Krogen
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment