Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5990 [Umbrella] Follow-up (phase-2) FairScheduler preemption improvements
  3. YARN-6193

FairScheduler might not trigger preemption when using DRF

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.8.0
    • 2.9.0, 3.0.0-alpha4
    • fairscheduler
    • None
    • Reviewed

    Description

      FSAppAttempt#canContainerBePreempted verifies preempting a container doesn't lead to new starvation (Resources.fitsIn). When using DRF, this leads to verifying both resources instead of just the dominant resource. Ideally, the check should be based on policy.

      Note that current implementation of DominantResourceFairnessPolicy#checkIfUsageOverFairShare is broken.

      Attachments

        1. YARN-6193.003.patch
          5 kB
          Karthik Kambatla
        2. YARN-6193.002.patch
          21 kB
          Karthik Kambatla
        3. YARN-6193.001.patch
          16 kB
          Karthik Kambatla
        4. YARN-6193.000.patch
          16 kB
          Karthik Kambatla

        Issue Links

          is blocked by

          Sub-task - The sub-task of the issue YARN-6163 FS Preemption is a trickle for severely starved applications

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              kasha Karthik Kambatla
              kasha Karthik Kambatla
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: