Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4866

FairScheduler: AMs can consume all vcores leading to a livelock when using FAIR policy

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.8.0
    • 2.9.0, 3.0.0-alpha1
    • fairscheduler
    • None
    • Reviewed

    Description

      The maxAMShare uses the queue's policy for enforcing limits. When using FAIR policy, this considers only memory. If there are fewer vcores on the cluster, the AMs can end up taking all the vcores leading to a livelock.

      Attachments

        1. YARN-4866.001.patch
          5 kB
          Yufei Gu
        2. YARN-4866.002.patch
          5 kB
          Yufei Gu
        3. YARN-4866.003.patch
          6 kB
          Yufei Gu
        4. YARN-4866.004.patch
          6 kB
          Yufei Gu
        5. YARN-4866.005.patch
          6 kB
          Yufei Gu
        6. YARN-4866.006.patch
          7 kB
          Yufei Gu
        7. YARN-4866.007.patch
          7 kB
          Yufei Gu
        8. YARN-4866.008.patch
          8 kB
          Yufei Gu
        9. YARN-4866.009.patch
          9 kB
          Yufei Gu

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. YARN-3621 FairScheduler doesn't count AM vcores towards max-share

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              yufeigu Yufei Gu
              kasha Karthik Kambatla
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: