Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9515

Non-exclusive labels do not respect user-limit-factor/max-capacity

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.8.5
    • Fix Version/s: None
    • Component/s: capacity scheduler
    • Labels:
      None

      Description

      When using a cluster with non-exclusive labels. Queues will only respect user-limit-factor for allocations to the default partition. Non-exclusive label allocations will not consider these factors allowing the labeled partitions capacity to be completely used up.

      To reproduce this, consider this example.

      Cluster contains 1 non-exclusive nodelabel: APPMASTER

      "test" queue has access to all labels: *

      "test" queue has capacity/max-capacity: 20

      "test" queue has user-limit-factor: 0.1

      Job is submitted to "test" queue with label: APPMASTER only on its appMaster container request, and no-label for all other containers.

      user-limit-factor and capacity will not be respected for the job on allocations to the APPMASTER partition causing the single job to attempt to use the entire capacity of the APPMASTER partition through non-exclusive allocations.

       

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              bdscheller Brandon Scheller
            • Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: