Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-10496 [Umbrella] Support Flexible Auto Queue Creation in Capacity Scheduler
  3. YARN-10531

Be able to disable user limit factor for CapacityScheduler Leaf Queue

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.4.0
    • Component/s: None
    • Labels:
      None

      Description

      User limit factor is used to define max cap of how much resource can be consumed by single user.

      Under Auto Queue Creation context, it doesn't make much sense to set user limit factor, because initially every queue will set weight to 1.0, we want user can consume more resource if possible. It is hard to pre-determine how to set up user limit factor. So it makes more sense to add a new value (like -1) to indicate we will disable user limit factor

      Logic need to be changed is below:

      (Inside LeafQueue.java)

          Resource maxUserLimit = Resources.none();
          if (schedulingMode == SchedulingMode.RESPECT_PARTITION_EXCLUSIVITY) {
            maxUserLimit = Resources.multiplyAndRoundDown(queueCapacity,
                getUserLimitFactor());
          } else if (schedulingMode == SchedulingMode.IGNORE_PARTITION_EXCLUSIVITY) {
            maxUserLimit = partitionResource;
          }
      

        Attachments

        1. YARN-10531.001.patch
          12 kB
          Qi Zhu
        2. YARN-10531.002.patch
          13 kB
          Qi Zhu
        3. YARN-10531.003.patch
          17 kB
          Qi Zhu
        4. YARN-10531.004.patch
          17 kB
          Qi Zhu
        5. YARN-10531.005.patch
          17 kB
          Qi Zhu
        6. YARN-10531.006.patch
          17 kB
          Qi Zhu
        7. YARN-10531.007.patch
          17 kB
          Qi Zhu

          Activity

            People

            • Assignee:
              zhuqi Qi Zhu
              Reporter:
              wangda Wangda Tan
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: