[YARN-3990] AsyncDispatcher may overloaded with RMAppNodeUpdateEvent when Node is connected/disconnected - ASF JIRA

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Closed
Priority: Critical
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 2.6.1, 2.8.0, 2.7.2, 3.0.0-alpha1
Component/s: resourcemanager
Labels:
- 2.6.1-candidate

Target Version/s:

2.7.2
Hadoop Flags:

Reviewed

Description

Whenever node is added or removed, NodeListManager sends RMAppNodeUpdateEvent to all the applications that are in the rmcontext. But for finished/killed/failed applications it is not required to send these events. Additional check for wheather app is finished/killed/failed would minimizes the unnecessary events

  public void handle(NodesListManagerEvent event) {
    RMNode eventNode = event.getNode();
    switch (event.getType()) {
    case NODE_UNUSABLE:
      LOG.debug(eventNode + " reported unusable");
      unusableRMNodesConcurrentSet.add(eventNode);
      for(RMApp app: rmContext.getRMApps().values()) {
        this.rmContext
            .getDispatcher()
            .getEventHandler()
            .handle(
                new RMAppNodeUpdateEvent(app.getApplicationId(), eventNode,
                    RMAppNodeUpdateType.NODE_UNUSABLE));
      }
      break;
    case NODE_USABLE:
      if (unusableRMNodesConcurrentSet.contains(eventNode)) {
        LOG.debug(eventNode + " reported usable");
        unusableRMNodesConcurrentSet.remove(eventNode);
      }
      for (RMApp app : rmContext.getRMApps().values()) {
        this.rmContext
            .getDispatcher()
            .getEventHandler()
            .handle(
                new RMAppNodeUpdateEvent(app.getApplicationId(), eventNode,
                    RMAppNodeUpdateType.NODE_USABLE));
      }
      break;
    default:
      LOG.error("Ignoring invalid eventtype " + event.getType());
    }
  }