[YARN-7903] Method getStarvedResourceRequests() only consider the first encountered resource

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.1.0
• Component/s: fairscheduler
• Assignee: Unassigned
• Reporter: Yufei Gu
• Created: 07/Feb/18 19:08
• Updated: 23/Feb/24 14:26

Description

We need to specify rack and ANY while submitting a node local resource request, as YARN-7561 discussed. For example:

    ResourceRequest nodeRequest =
            createResourceRequest(GB, node1.getHostName(), 1, 1, false);
    ResourceRequest rackRequest =
            createResourceRequest(GB, node1.getRackName(), 1, 1, false);
    ResourceRequest anyRequest =
            createResourceRequest(GB, ResourceRequest.ANY, 1, 1, false);

    List<ResourceRequest> resourceRequests =
            Arrays.asList(nodeRequest, rackRequest, anyRequest);

However, method getStarvedResourceRequests() only consider the first encountered resource, which most likely is ResourceRequest.ANY. That's a mismatch for locality request.

Issue Links

• blocks: YARN-7910 Fix TODO in TestFairSchedulerPreemption#testRelaxLocalityToNotPreemptAM
• relates to: YARN-6956 preemption may only consider resource requests for one node
• relates to: YARN-7655 Avoid AM preemption caused by RRs for specific nodes or racks