Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4866

FairScheduler: AMs can consume all vcores leading to a livelock when using FAIR policy

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.8.0
    • 2.9.0, 3.0.0-alpha1
    • fairscheduler
    • None
    • Reviewed

    Description

      The maxAMShare uses the queue's policy for enforcing limits. When using FAIR policy, this considers only memory. If there are fewer vcores on the cluster, the AMs can end up taking all the vcores leading to a livelock.

      Attachments

        1. YARN-4866.001.patch
          5 kB
          Yufei Gu
        2. YARN-4866.002.patch
          5 kB
          Yufei Gu
        3. YARN-4866.003.patch
          6 kB
          Yufei Gu
        4. YARN-4866.004.patch
          6 kB
          Yufei Gu
        5. YARN-4866.005.patch
          6 kB
          Yufei Gu
        6. YARN-4866.006.patch
          7 kB
          Yufei Gu
        7. YARN-4866.007.patch
          7 kB
          Yufei Gu
        8. YARN-4866.008.patch
          8 kB
          Yufei Gu
        9. YARN-4866.009.patch
          9 kB
          Yufei Gu

        Issue Links

        duplicates

        Bug - A problem which impairs or prevents the functions of the product. YARN-3621 FairScheduler doesn't count AM vcores towards max-share

        • Major - Major loss of function.
        • Resolved

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            yufeigu Yufei Gu
            kasha Karthik Kambatla
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment