Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5542 Scheduling of opportunistic containers
  3. YARN-7610

Extend Distributed Shell to support launching job with opportunistic containers

    Details

    • Hadoop Flags:
      Reviewed

      Description

      Per doc in https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/OpportunisticContainers.html#Running_a_Sample_Job, user can run some of PI job mappers as O containers. Similarly, propose to extend distributed shell to support specifying the container type, it will be very helpful for testing. Propose to add following argument

      $./bin/yarn org.apache.hadoop.yarn.applications.distributedshell.Client
      -container_type <arg>                          Container execution type,
                                                      GUARANTEED or
                                                      OPPORTUNISTIC
      

      Implication: all containers in a distributed shell job will be launching as user-specified container type (except for AM), if not given, default type is GUARANTEED. AM is always launched as GUARANTEED container.

        Attachments

        1. YARN-7610.001.patch
          10 kB
          Weiwei Yang
        2. YARN-7610.002.patch
          24 kB
          Weiwei Yang
        3. added_doc.png
          311 kB
          Weiwei Yang
        4. outline_compare.png
          326 kB
          Weiwei Yang
        5. YARN-7610.003.patch
          25 kB
          Weiwei Yang
        6. YARN-7610.004.patch
          27 kB
          Weiwei Yang
        7. YARN-7610.005.patch
          25 kB
          Weiwei Yang

          Activity

            People

            • Assignee:
              cheersyang Weiwei Yang
              Reporter:
              cheersyang Weiwei Yang
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: