[HDFS-13272] DataNodeHttpServer to have configurable HttpServer2 threads

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.9.2
• Component/s: datanode
• Target Version/s: 2.10.0
• Hadoop Flags: Reviewed
• Assignee: Erik Krogen
• Reporter: Erik Krogen
• Created: 13/Mar/18 16:57
• Updated: 23/May/18 19:11
• Resolved: 10/May/18 22:37

Description

In HDFS-7279, the Jetty server on the DataNode was hard-coded to use 10 threads. In addition to the possibility of this being too few threads, it is much higher than necessary in resource constrained environments such as MiniDFSCluster. To avoid compatibility issues, rather than using HttpServer2#HTTP_MAX_THREADS directly, we can introduce a new configuration for the DataNode's thread pool size.

Issue Links

• is depended upon by: HDFS-13265 MiniDFSCluster should set reasonable defaults to reduce resource consumption
• relates to: HDFS-13493 Reduce the HttpServer2 thread count on DataNodes