[YARN-7610] Extend Distributed Shell to support launching job with opportunistic containers

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.1.0, 3.0.1
• Component/s: applications/distributed-shell
• Hadoop Flags: Reviewed
• Assignee: Weiwei Yang
• Reporter: Weiwei Yang
• Created: 05/Dec/17 08:21
• Updated: 18/Dec/17 03:20
• Resolved: 06/Dec/17 11:48

Description

Per doc in https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/OpportunisticContainers.html#Running_a_Sample_Job, user can run some of PI job mappers as O containers. Similarly, propose to extend distributed shell to support specifying the container type, it will be very helpful for testing. Propose to add following argument

$./bin/yarn org.apache.hadoop.yarn.applications.distributedshell.Client
-container_type <arg>                          Container execution type,
                                                GUARANTEED or
                                                OPPORTUNISTIC

Implication: all containers in a distributed shell job will be launching as user-specified container type (except for AM), if not given, default type is GUARANTEED. AM is always launched as GUARANTEED container.