Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-1913

With Fair Scheduler, cluster can logjam when all resources are consumed by AMs

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.3.0
    • 2.5.0
    • scheduler
    • None

    Description

      It's possible to deadlock a cluster by submitting many applications at once, and have all cluster resources taken up by AMs.

      One solution is for the scheduler to limit resources taken up by AMs, as a percentage of total cluster resources, via a "maxApplicationMasterShare" config.

      Attachments

        1. YARN-1913.patch
          40 kB
          Wei Yan
        2. YARN-1913.patch
          40 kB
          Wei Yan
        3. YARN-1913.patch
          40 kB
          Wei Yan
        4. YARN-1913.patch
          36 kB
          Wei Yan
        5. YARN-1913.patch
          36 kB
          Wei Yan
        6. YARN-1913.patch
          87 kB
          Wei Yan
        7. YARN-1913.patch
          86 kB
          Wei Yan
        8. YARN-1913.patch
          34 kB
          Wei Yan
        9. YARN-1913.patch
          35 kB
          Wei Yan
        10. YARN-1913.patch
          35 kB
          Wei Yan
        11. YARN-1913.patch
          38 kB
          Wei Yan

        Activity

          People

            ywskycn Wei Yan
            bcwalrus bc Wong
            Votes:
            0 Vote for this issue
            Watchers:
            21 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: