Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-10888 [Umbrella] New capacity modes for CS
  3. YARN-9936

Support vector of capacity percentages in Capacity Scheduler configuration

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Invalid
    • None
    • None
    • capacity scheduler
    • None

    Description

      Currently, the Capacity Scheduler queue configuration supports two ways to set queue capacity.

      • In percentage of all available resources as a float ( eg. 25.0 ) means 25% of the resources of its parent queue for all resource types equally (eg. 25% of all memory, 25% of all CPU cores, and 25% of all available GPU in the cluster) The percentages of all queues has to add up to 100%.
      • In an absolute amount of resources ( e.g. memory=4GB,vcores=20,yarn.io/gpu=4 ). The amount of all resources in the queues has to be less than or equal to all resources in the cluster.Actually, the above is not supported, we only support memory and vcores now in absolute mode, we should extend in  YARN-10503.

      Apart from these two already existing ways, there is a demand to add capacity percentage of each available resource type separately. (eg. memory=20%,vcores=40%,yarn.io/gpu=100%).
      At the same time, a similar concept should be included with queues maximum-capacity as well.

      Attachments

        Issue Links

          Activity

            People

              gandras Andras Gyori
              zsiegl Zoltan Siegl
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: