Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4866

FairScheduler: AMs can consume all vcores leading to a livelock when using FAIR policy

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.9.0, 3.0.0-alpha1
    • Component/s: fairscheduler
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      The maxAMShare uses the queue's policy for enforcing limits. When using FAIR policy, this considers only memory. If there are fewer vcores on the cluster, the AMs can end up taking all the vcores leading to a livelock.

        Attachments

        1. YARN-4866.001.patch
          5 kB
          Yufei Gu
        2. YARN-4866.002.patch
          5 kB
          Yufei Gu
        3. YARN-4866.003.patch
          6 kB
          Yufei Gu
        4. YARN-4866.004.patch
          6 kB
          Yufei Gu
        5. YARN-4866.005.patch
          6 kB
          Yufei Gu
        6. YARN-4866.006.patch
          7 kB
          Yufei Gu
        7. YARN-4866.007.patch
          7 kB
          Yufei Gu
        8. YARN-4866.008.patch
          8 kB
          Yufei Gu
        9. YARN-4866.009.patch
          9 kB
          Yufei Gu

          Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. YARN-3621 FairScheduler doesn't count AM vcores towards max-share

          • Major - Major loss of function.
          • Resolved

            Activity

              People

              • Assignee:
                yufeigu Yufei Gu
                Reporter:
                kasha Karthik Kambatla
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: