Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-10396

Max applications calculation per queue disregards queue level settings in absolute mode

    XMLWordPrintableJSON

    Details

    • Hadoop Flags:
      Reviewed

      Description

      Looking at the following code in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.java#L1126

      int maxApplications = (int) (conf.getMaximumSystemApplications()
          * childQueue.getQueueCapacities().getAbsoluteCapacity(label));
      leafQueue.setMaxApplications(maxApplications);

      In Absolute Resources mode setting the number of maximum applications on queue level gets overridden with the system level setting scaled down to the available resources. This means that the only way to set the maximum number of applications is to change the queue's resource pool. This line should consider the queue's {{yarn.scheduler.capacity.{queuepath}.maximum-applications }}setting.

        Attachments

        1. YARN-10396.001.patch
          2 kB
          Benjamin Teke
        2. YARN-10396.002.patch
          11 kB
          Benjamin Teke
        3. YARN-10396.003.patch
          12 kB
          Benjamin Teke
        4. YARN-10396.branch-3.1.001.patch
          12 kB
          Benjamin Teke
        5. YARN-10396.branch-3.2.001.patch
          12 kB
          Benjamin Teke

          Activity

            People

            • Assignee:
              bteke Benjamin Teke
              Reporter:
              bteke Benjamin Teke
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: