Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-10496 [Umbrella] Support Flexible Auto Queue Creation in Capacity Scheduler
  3. YARN-10531

Be able to disable user limit factor for CapacityScheduler Leaf Queue

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.4.0
    • 3.4.0
    • capacity scheduler
    • None
    • Reviewed

    Description

      User limit factor is used to define max cap of how much resource can be consumed by single user.

      Under Auto Queue Creation context, it doesn't make much sense to set user limit factor, because initially every queue will set weight to 1.0, we want user can consume more resource if possible. It is hard to pre-determine how to set up user limit factor. So it makes more sense to add a new value (like -1) to indicate we will disable user limit factor

      Logic need to be changed is below:

      (Inside LeafQueue.java)

          Resource maxUserLimit = Resources.none();
          if (schedulingMode == SchedulingMode.RESPECT_PARTITION_EXCLUSIVITY) {
            maxUserLimit = Resources.multiplyAndRoundDown(queueCapacity,
                getUserLimitFactor());
          } else if (schedulingMode == SchedulingMode.IGNORE_PARTITION_EXCLUSIVITY) {
            maxUserLimit = partitionResource;
          }
      

      Attachments

        1. YARN-10531.001.patch
          12 kB
          Qi Zhu
        2. YARN-10531.002.patch
          13 kB
          Qi Zhu
        3. YARN-10531.003.patch
          17 kB
          Qi Zhu
        4. YARN-10531.004.patch
          17 kB
          Qi Zhu
        5. YARN-10531.005.patch
          17 kB
          Qi Zhu
        6. YARN-10531.006.patch
          17 kB
          Qi Zhu
        7. YARN-10531.007.patch
          17 kB
          Qi Zhu

        Activity

          People

            zhuqi Qi Zhu
            wangda Wangda Tan
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: