Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5542 Scheduling of opportunistic containers
  3. YARN-7610

Extend Distributed Shell to support launching job with opportunistic containers

    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      Per doc in https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/OpportunisticContainers.html#Running_a_Sample_Job, user can run some of PI job mappers as O containers. Similarly, propose to extend distributed shell to support specifying the container type, it will be very helpful for testing. Propose to add following argument

      $./bin/yarn org.apache.hadoop.yarn.applications.distributedshell.Client
      -container_type <arg>                          Container execution type,
                                                      GUARANTEED or
                                                      OPPORTUNISTIC
      

      Implication: all containers in a distributed shell job will be launching as user-specified container type (except for AM), if not given, default type is GUARANTEED. AM is always launched as GUARANTEED container.

      Attachments

        1. YARN-7610.005.patch
          25 kB
          Weiwei Yang
        2. YARN-7610.004.patch
          27 kB
          Weiwei Yang
        3. YARN-7610.003.patch
          25 kB
          Weiwei Yang
        4. YARN-7610.002.patch
          24 kB
          Weiwei Yang
        5. YARN-7610.001.patch
          10 kB
          Weiwei Yang
        6. outline_compare.png
          326 kB
          Weiwei Yang
        7. added_doc.png
          311 kB
          Weiwei Yang

        Activity

          People

            cheersyang Weiwei Yang
            cheersyang Weiwei Yang
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: