Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-10396

Max applications calculation per queue disregards queue level settings in absolute mode

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.2.2, 3.4.0, 3.3.1
    • capacity scheduler
    • None
    • Reviewed

    Description

      Looking at the following code in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.java#L1126

      int maxApplications = (int) (conf.getMaximumSystemApplications()
          * childQueue.getQueueCapacities().getAbsoluteCapacity(label));
      leafQueue.setMaxApplications(maxApplications);

      In Absolute Resources mode, setting the number of maximum applications on queue level gets overridden with the system level setting scaled down to the available resources. This means that the only way to set the maximum number of applications is to change the queue's resource pool. This line should consider the queue's {{yarn.scheduler.capacity.{queuepath}.maximum-applications }}setting.

      Attachments

        1. YARN-10396.001.patch
          2 kB
          Benjamin Teke
        2. YARN-10396.002.patch
          11 kB
          Benjamin Teke
        3. YARN-10396.003.patch
          12 kB
          Benjamin Teke
        4. YARN-10396.branch-3.1.001.patch
          12 kB
          Benjamin Teke
        5. YARN-10396.branch-3.2.001.patch
          12 kB
          Benjamin Teke

        Activity

          People

            bteke Benjamin Teke
            bteke Benjamin Teke
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: