Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4212

FairScheduler: Can't create a DRF queue under a FAIR policy queue

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.9.0, 3.0.0-alpha4
    • None
    • Reviewed

    Description

      The Fair Scheduler, while performing a recomputeShares() during an update() call, uses the parent queues policy to distribute shares to its children.

      If the parent queues policy is 'fair', it only computes weight for memory and sets the vcores fair share of its children to 0.

      Assuming a situation where we have 1 parent queue with policy 'fair' and multiple leaf queues with policy 'drf', Any app submitted to the child queues with vcore requirement > 1 will always be above fairshare, since during the recomputeShare process, the child queues were all assigned 0 for fairshare vcores.

      Attachments

        1. YARN-4212.1.patch
          7 kB
          Arun Suresh
        2. YARN-4212.011.patch
          38 kB
          Yufei Gu
        3. YARN-4212.010.patch
          38 kB
          Yufei Gu
        4. YARN-4212.009.patch
          38 kB
          Yufei Gu
        5. YARN-4212.008.patch
          34 kB
          Yufei Gu
        6. YARN-4212.007.patch
          25 kB
          Yufei Gu
        7. YARN-4212.006.patch
          16 kB
          Yufei Gu
        8. YARN-4212.005.patch
          12 kB
          Yufei Gu
        9. YARN-4212.004.patch
          20 kB
          Yufei Gu
        10. YARN-4212.003.patch
          9 kB
          Yufei Gu
        11. YARN-4212.002.patch
          8 kB
          Yufei Gu

        Issue Links

          Activity

            People

              yufeigu Yufei Gu
              asuresh Arun Suresh
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: