Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-10888 [Umbrella] New capacity modes for CS
  3. YARN-9936

Support vector of capacity percentages in Capacity Scheduler configuration

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: capacity scheduler
    • Labels:
      None
    • Target Version/s:

      Description

      Currently, the Capacity Scheduler queue configuration supports two ways to set queue capacity.

      • In percentage of all available resources as a float ( eg. 25.0 ) means 25% of the resources of its parent queue for all resource types equally (eg. 25% of all memory, 25% of all CPU cores, and 25% of all available GPU in the cluster) The percentages of all queues has to add up to 100%.
      • In an absolute amount of resources ( e.g. memory=4GB,vcores=20,yarn.io/gpu=4 ). The amount of all resources in the queues has to be less than or equal to all resources in the cluster.Actually, the above is not supported, we only support memory and vcores now in absolute mode, we should extend in  YARN-10503.

      Apart from these two already existing ways, there is a demand to add capacity percentage of each available resource type separately. (eg. memory=20%,vcores=40%,yarn.io/gpu=100%).
      At the same time, a similar concept should be included with queues maximum-capacity as well.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                gandras Andras Gyori
                Reporter:
                zsiegl Zoltan Siegl
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated: