[YARN-10705] Misleading DEBUG log for container assignment needs to be removed when the container is actually reserved, not assigned in FairScheduler

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: yarn
• Hadoop Flags: Reviewed
• Assignee: Siddharth Ahuja
• Reporter: Siddharth Ahuja
• Created: 19/Mar/21 16:40
• Updated: 23/Apr/21 15:41
• Resolved: 23/Apr/21 15:41

Description

Following DEBUG logs are logged if a container reservation is made when a node has been offered to the queue in FairScheduler:

2021-02-10 07:33:55,049 DEBUG org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt: application_1610442362681_2607's resource request is reserved.
2021-02-10 07:33:55,049 DEBUG org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue: Assigned container in queue:root.pj_dc_pe container:<memory:-1, vCores:0>

The latter log from above seems to indicate a bad container assignment with <memory:-1, vCores:0> resource allocation, whereas, in actual, it is a bad log which shouldn't have been logged in the first place.

This log comes from [1] after an application attempt with an unmet demand is checked for container assignment/reservation.

If the container for this app attempt is reserved on the node, then, it returns <memory:-1, vCores:0> from [2].

From [3]:

• If an assignment was made, returns the resources allocated to the
• container. If a reservation was made, returns
• FairScheduler.CONTAINER_RESERVED. If no assignment or reservation was
• made, returns an empty resource.

We are checking for the empty resource at [4], but not FairScheduler.CONTAINER_RESERVED before logging out a message for container assignment specifically which is incorrect.

Instead of:

      if (!assigned.equals(none())) {
        LOG.debug("Assigned container in queue:{} container:{}",
            getName(), assigned);
        break;
      }

it should be:

      // check if an assignment or a reservation was made.
      if (!assigned.equals(none())) {
        // only log container assignment if there is
        // an actual assignment, not a reservation.
        if (!assigned.equals(FairScheduler.CONTAINER_RESERVED)
                && LOG.isDebugEnabled()) {
          LOG.debug("Assigned container in queue:" + getName() + " " +
                        "container:" + assigned);
        }
        break;
      }

[1] https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSLeafQueue.java#L356
[2] https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java#L911
[3] https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSAppAttempt.java#L842
[4] https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FSLeafQueue.java#L355