[MAPREDUCE-6265] Make ContainerLauncherImpl.INITIAL_POOL_SIZE configurable to better control to launch/kill containers

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.7.0
• Component/s: mrv2
• Hadoop Flags: Reviewed
• Assignee: Zhihai Xu
• Reporter: Zhihai Xu
• Created: 23/Feb/15 08:46
• Updated: 24/Apr/15 23:15
• Resolved: 14/Mar/15 07:50

Description

make INITIAL_POOL_SIZE in ContainerLauncherImpl configurable to better control the thread pool size to launch/kill containers
Currently INITIAL_POOL_SIZE in ContainerLauncherImpl is hard-coded at

  protected static final int INITIAL_POOL_SIZE = 10;

We should make it configurable because the thread pool size will be decided by INITIAL_POOL_SIZE, limitOnPoolSize and number of node used by the AM.
Since we already made limitOnPoolSize configurable, it make senses to also make INITIAL_POOL_SIZE configurable to better manage the thread pool size.
We saw some issue due to the small thread pool size when some node is down. The recovery from a shutdown node take very long time due to all the ContainerLauncher threads are blocked by IPC client connection to the shutdown node.