Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4212

FairScheduler: Can't create a DRF queue under a FAIR policy queue

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha4
    • Component/s: None
    • Labels:
    • Hadoop Flags:
      Reviewed

      Description

      The Fair Scheduler, while performing a recomputeShares() during an update() call, uses the parent queues policy to distribute shares to its children.

      If the parent queues policy is 'fair', it only computes weight for memory and sets the vcores fair share of its children to 0.

      Assuming a situation where we have 1 parent queue with policy 'fair' and multiple leaf queues with policy 'drf', Any app submitted to the child queues with vcore requirement > 1 will always be above fairshare, since during the recomputeShare process, the child queues were all assigned 0 for fairshare vcores.

        Attachments

        1. YARN-4212.002.patch
          8 kB
          Yufei Gu
        2. YARN-4212.003.patch
          9 kB
          Yufei Gu
        3. YARN-4212.004.patch
          20 kB
          Yufei Gu
        4. YARN-4212.005.patch
          12 kB
          Yufei Gu
        5. YARN-4212.006.patch
          16 kB
          Yufei Gu
        6. YARN-4212.007.patch
          25 kB
          Yufei Gu
        7. YARN-4212.008.patch
          34 kB
          Yufei Gu
        8. YARN-4212.009.patch
          38 kB
          Yufei Gu
        9. YARN-4212.010.patch
          38 kB
          Yufei Gu
        10. YARN-4212.011.patch
          38 kB
          Yufei Gu
        11. YARN-4212.1.patch
          7 kB
          Arun Suresh

          Issue Links

            Activity

              People

              • Assignee:
                yufeigu Yufei Gu
                Reporter:
                asuresh Arun Suresh
              • Votes:
                0 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: