[YARN-3990] AsyncDispatcher may overloaded with RMAppNodeUpdateEvent when Node is connected/disconnected

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 2.6.1, 2.8.0, 2.7.2, 3.0.0-alpha1
• Component/s: resourcemanager
• Labels: 2.6.1-candidate
• Target Version/s: 2.7.2
• Hadoop Flags: Reviewed
• Assignee: Bibin Chundatt
• Reporter: Rohith Sharma K S
• Created: 29/Jul/15 06:43
• Updated: 28/Jul/21 12:33
• Resolved: 31/Jul/15 17:40

Description

Whenever node is added or removed, NodeListManager sends RMAppNodeUpdateEvent to all the applications that are in the rmcontext. But for finished/killed/failed applications it is not required to send these events. Additional check for wheather app is finished/killed/failed would minimizes the unnecessary events

  public void handle(NodesListManagerEvent event) {
    RMNode eventNode = event.getNode();
    switch (event.getType()) {
    case NODE_UNUSABLE:
      LOG.debug(eventNode + " reported unusable");
      unusableRMNodesConcurrentSet.add(eventNode);
      for(RMApp app: rmContext.getRMApps().values()) {
        this.rmContext
            .getDispatcher()
            .getEventHandler()
            .handle(
                new RMAppNodeUpdateEvent(app.getApplicationId(), eventNode,
                    RMAppNodeUpdateType.NODE_UNUSABLE));
      }
      break;
    case NODE_USABLE:
      if (unusableRMNodesConcurrentSet.contains(eventNode)) {
        LOG.debug(eventNode + " reported usable");
        unusableRMNodesConcurrentSet.remove(eventNode);
      }
      for (RMApp app : rmContext.getRMApps().values()) {
        this.rmContext
            .getDispatcher()
            .getEventHandler()
            .handle(
                new RMAppNodeUpdateEvent(app.getApplicationId(), eventNode,
                    RMAppNodeUpdateType.NODE_USABLE));
      }
      break;
    default:
      LOG.error("Ignoring invalid eventtype " + event.getType());
    }
  }

Issue Links

• is related to: YARN-3979 Am in ResourceLocalizationService hang 10 min cause RM kill AM
• relates to: YARN-9618 NodesListManager event improvement