Hadoop YARN
  1. Hadoop YARN
  2. YARN-1913

With Fair Scheduler, cluster can logjam when all resources are consumed by AMs

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.3.0
    • Fix Version/s: 2.5.0
    • Component/s: scheduler
    • Labels:
      None

      Description

      It's possible to deadlock a cluster by submitting many applications at once, and have all cluster resources taken up by AMs.

      One solution is for the scheduler to limit resources taken up by AMs, as a percentage of total cluster resources, via a "maxApplicationMasterShare" config.

      1. YARN-1913.patch
        40 kB
        Wei Yan
      2. YARN-1913.patch
        40 kB
        Wei Yan
      3. YARN-1913.patch
        40 kB
        Wei Yan
      4. YARN-1913.patch
        36 kB
        Wei Yan
      5. YARN-1913.patch
        36 kB
        Wei Yan
      6. YARN-1913.patch
        87 kB
        Wei Yan
      7. YARN-1913.patch
        86 kB
        Wei Yan
      8. YARN-1913.patch
        34 kB
        Wei Yan
      9. YARN-1913.patch
        35 kB
        Wei Yan
      10. YARN-1913.patch
        35 kB
        Wei Yan
      11. YARN-1913.patch
        38 kB
        Wei Yan

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Wei Yan
            Reporter:
            bc Wong
          • Votes:
            0 Vote for this issue
            Watchers:
            20 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development