Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-1913

With Fair Scheduler, cluster can logjam when all resources are consumed by AMs

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.3.0
    • Fix Version/s: 2.5.0
    • Component/s: scheduler
    • Labels:
      None

      Description

      It's possible to deadlock a cluster by submitting many applications at once, and have all cluster resources taken up by AMs.

      One solution is for the scheduler to limit resources taken up by AMs, as a percentage of total cluster resources, via a "maxApplicationMasterShare" config.

        Attachments

        1. YARN-1913.patch
          38 kB
          Wei Yan
        2. YARN-1913.patch
          35 kB
          Wei Yan
        3. YARN-1913.patch
          35 kB
          Wei Yan
        4. YARN-1913.patch
          34 kB
          Wei Yan
        5. YARN-1913.patch
          86 kB
          Wei Yan
        6. YARN-1913.patch
          87 kB
          Wei Yan
        7. YARN-1913.patch
          36 kB
          Wei Yan
        8. YARN-1913.patch
          36 kB
          Wei Yan
        9. YARN-1913.patch
          40 kB
          Wei Yan
        10. YARN-1913.patch
          40 kB
          Wei Yan
        11. YARN-1913.patch
          40 kB
          Wei Yan

          Activity

            People

            • Assignee:
              ywskycn Wei Yan
              Reporter:
              bcwalrus bc Wong
            • Votes:
              0 Vote for this issue
              Watchers:
              21 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: