[YARN-8059] Resource type is ignored when FS decide to preempt

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.0.0
• Component/s: fairscheduler
• Assignee: Szilard Nemeth
• Reporter: Yufei Gu
• Created: 21/Mar/18 20:24
• Updated: 26/Feb/20 05:29

Description

Method Fairscheduler#shouldAttemptPreemption doesn't consider resources other than vcore and memory. We may need to rethink it in the resource type scenario. cc [e-mail], wilfreds and snemeth.

    if (context.isPreemptionEnabled()) {
      return (context.getPreemptionUtilizationThreshold() < Math.max(
          (float) rootMetrics.getAllocatedMB() /
              getClusterResource().getMemorySize(),
          (float) rootMetrics.getAllocatedVirtualCores() /
              getClusterResource().getVirtualCores()));
    }

Issue Links

• depends upon: YARN-9019 Ratio calculation of ResourceCalculator implementations could return NaN
• depends upon: YARN-9025 TestFairScheduler#testChildMaxResources is flaky
• depends upon: YARN-8842 Expose metrics for custom resource types in QueueMetrics
• depends upon: YARN-8750 Refactor TestQueueMetrics
• is blocked by: YARN-9047 FairScheduler: default resource calculator is not resource type aware
• is depended upon by: YARN-5106 Provide a builder interface for FairScheduler allocations for use in tests
• relates to: YARN-8842 Expose metrics for custom resource types in QueueMetrics