Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9698 [Umbrella] Tools to help migration from Fair Scheduler to Capacity Scheduler
  3. YARN-10257

FS-CS converter: skip increment properties for mem/vcores and fix DRF check

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.3.0, 3.4.0
    • 3.3.0, 3.4.0
    • fairscheduler, fs-cs
    • None
    • Reviewed

    Description

      Two issues have been discovered during fs2cs testing:

      1. The conversion of allocation increment properties are not needed:

      yarn.scheduler.increment-allocation-mb
      yarn.scheduler.increment-allocation-vcores
      yarn.resource-types.memory-mb.increment-allocation
      yarn.resource-types.vcores.increment-allocation

      2. The following piece of code is incorrect - the default scheduling policy can be different from DRF, which is a problem if DRF is used everywhere else:

        private boolean isDrfUsed(FairScheduler fs) {
          FSQueue rootQueue = fs.getQueueManager().getRootQueue();
          AllocationConfiguration allocConf = fs.getAllocationConfiguration();
      
          String defaultPolicy = allocConf.getDefaultSchedulingPolicy().getName();
      
          if (DominantResourceFairnessPolicy.NAME.equals(defaultPolicy)) {
            return true;
          } else {
            return isDrfUsedOnQueueLevel(rootQueue);
          }
        }
      

      Attachments

        1. YARN-10257-002.patch
          11 kB
          Peter Bacsko
        2. YARN-10257-001.patch
          11 kB
          Peter Bacsko

        Activity

          People

            pbacsko Peter Bacsko
            pbacsko Peter Bacsko
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: