Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3446

FairScheduler headroom calculation should exclude nodes in the blacklist

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: fairscheduler
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      FairScheduler HeadRoom calculation should exclude nodes in the blacklist.
      MRAppMaster does not preempt the reducers because for Reducer preemption calculation, headRoom is considering blacklisted nodes. This makes jobs to hang forever(ResourceManager does not assign any new containers on blacklisted nodes but availableResource AM get from RM includes blacklisted nodes available resource).
      This issue is similar as YARN-1680 which is for Capacity Scheduler.

        Attachments

        1. YARN-3446.000.patch
          11 kB
          zhihai xu
        2. YARN-3446.001.patch
          10 kB
          zhihai xu
        3. YARN-3446.002.patch
          11 kB
          zhihai xu
        4. YARN-3446.003.patch
          17 kB
          zhihai xu
        5. YARN-3446.004.patch
          16 kB
          zhihai xu
        6. YARN-3446.005.patch
          16 kB
          zhihai xu
        7. YARN-3446-branch-2.8.txt
          18 kB
          Vinod Kumar Vavilapalli

          Issue Links

            Activity

              People

              • Assignee:
                zxu zhihai xu
                Reporter:
                zxu zhihai xu
              • Votes:
                0 Vote for this issue
                Watchers:
                12 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: