[YARN-10257] FS-CS converter: skip increment properties for mem/vcores and fix DRF check

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.0, 3.4.0
• Fix Version/s: 3.3.0, 3.4.0
• Component/s: fairscheduler, fs-cs
• Target Version/s: 3.3.0, 3.4.0
• Hadoop Flags: Reviewed
• Assignee: Peter Bacsko
• Reporter: Peter Bacsko
• Created: 03/May/20 17:21
• Updated: 12/Feb/24 00:08
• Resolved: 05/May/20 14:02

Description

Two issues have been discovered during fs2cs testing:

1. The conversion of allocation increment properties are not needed:

yarn.scheduler.increment-allocation-mb
yarn.scheduler.increment-allocation-vcores
yarn.resource-types.memory-mb.increment-allocation
yarn.resource-types.vcores.increment-allocation

2. The following piece of code is incorrect - the default scheduling policy can be different from DRF, which is a problem if DRF is used everywhere else:

  private boolean isDrfUsed(FairScheduler fs) {
    FSQueue rootQueue = fs.getQueueManager().getRootQueue();
    AllocationConfiguration allocConf = fs.getAllocationConfiguration();

    String defaultPolicy = allocConf.getDefaultSchedulingPolicy().getName();

    if (DominantResourceFairnessPolicy.NAME.equals(defaultPolicy)) {
      return true;
    } else {
      return isDrfUsedOnQueueLevel(rootQueue);
    }
  }