Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-13272

DataNodeHttpServer to have configurable HttpServer2 threads

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.9.2
    • datanode
    • None
    • Reviewed

    Description

      In HDFS-7279, the Jetty server on the DataNode was hard-coded to use 10 threads. In addition to the possibility of this being too few threads, it is much higher than necessary in resource constrained environments such as MiniDFSCluster. To avoid compatibility issues, rather than using HttpServer2#HTTP_MAX_THREADS directly, we can introduce a new configuration for the DataNode's thread pool size.

      Attachments

        1. HDFS-13272-branch-2.000.patch
          4 kB
          Erik Krogen
        2. HDFS-13272-branch-2.001.patch
          3 kB
          Erik Krogen
        3. testout-HDFS-13272.001
          161 kB
          Erik Krogen
        4. testout-branch-2
          387 kB
          Erik Krogen

        Issue Links

          is depended upon by

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-13265 MiniDFSCluster should set reasonable defaults to reduce resource consumption

          • Major - Major loss of function.
          • Patch Available
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-13493 Reduce the HttpServer2 thread count on DataNodes

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              xkrogen Erik Krogen
              xkrogen Erik Krogen
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: