Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-7561

Why hasContainerForNode() return false directly when there is no request of ANY locality without considering NODE_LOCAL and RACK_LOCAL?

    XMLWordPrintableJSON

Details

    • Task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.7.3, 3.1.0
    • None
    • fairscheduler
    • None

    Description

      I am studying the FairScheduler source cod of yarn 2.7.3.
      By the code of class FSAppAttempt:

        public boolean hasContainerForNode(Priority prio, FSSchedulerNode node) {
      
          ResourceRequest anyRequest = getResourceRequest(prio, ResourceRequest.ANY);  
          ResourceRequest rackRequest = getResourceRequest(prio, node.getRackName()); 
          ResourceRequest nodeRequest = getResourceRequest(prio, node.getNodeName()); 
          
          return
              // There must be outstanding requests at the given priority:
              anyRequest != null && anyRequest.getNumContainers() > 0 &&
                  // If locality relaxation is turned off at *-level, there must be a
                  // non-zero request for the node's rack:
                  (anyRequest.getRelaxLocality() ||
                      (rackRequest != null && rackRequest.getNumContainers() > 0)) &&
                  // If locality relaxation is turned off at rack-level, there must be a
                  // non-zero request at the node:
                  (rackRequest == null || rackRequest.getRelaxLocality() ||
                      (nodeRequest != null && nodeRequest.getNumContainers() > 0)) &&
                  // The requested container must be able to fit on the node:
                  Resources.lessThanOrEqual(RESOURCE_CALCULATOR, null,
                      anyRequest.getCapability(), node.getRMNode().getTotalCapability());
      }
      

      I really cannot understand why when there is no anyRequest , hasContainerForNode() return false directly without considering whether there is NODE_LOCAL or RACK_LOCAL requests.
      And , AppSchedulingInfo.allocateNodeLocal() and AppSchedulingInfo.allocateRackLocal() will also decrease the number of containers for ResourceRequest.ANY, this is another place where I feel confused.

      Really thanks for some prompt.

      Attachments

        Activity

          People

            Unassigned Unassigned
            wuchang1989 wuchang
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: