Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3920

FairScheduler: Limit node reservations to large containers

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • None
    • fairscheduler
    • None

    Description

      Reserving a node for a container was designed for preventing large containers from starvation from small requests that keep getting into a node. Today we let this be used even for a small container request. This has a huge impact on scheduling since we block other scheduling requests until that reservation is fulfilled. We should make this configurable so its impact can be minimized by limiting it for large container requests as originally intended.

      Attachments

        1. YARN-3920.005.patch
          14 kB
          Anubhav Dhoot
        2. YARN-3920.004.patch
          13 kB
          Anubhav Dhoot
        3. YARN-3920.004.patch
          13 kB
          Anubhav Dhoot
        4. YARN-3920.004.patch
          13 kB
          Anubhav Dhoot
        5. YARN-3920.004.patch
          15 kB
          Anubhav Dhoot
        6. yARN-3920.003.patch
          16 kB
          Anubhav Dhoot
        7. yARN-3920.002.patch
          12 kB
          Anubhav Dhoot
        8. yARN-3920.001.patch
          12 kB
          Anubhav Dhoot

        Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. YARN-7356 The fair scheduler reservation threshold should be documented in the fair scheduler docs

          • Major - Major loss of function.
          • Open

          Activity

            People

              adhoot Anubhav Dhoot
              adhoot Anubhav Dhoot
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: