Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3990

AsyncDispatcher may overloaded with RMAppNodeUpdateEvent when Node is connected/disconnected

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      Whenever node is added or removed, NodeListManager sends RMAppNodeUpdateEvent to all the applications that are in the rmcontext. But for finished/killed/failed applications it is not required to send these events. Additional check for wheather app is finished/killed/failed would minimizes the unnecessary events

        public void handle(NodesListManagerEvent event) {
          RMNode eventNode = event.getNode();
          switch (event.getType()) {
          case NODE_UNUSABLE:
            LOG.debug(eventNode + " reported unusable");
            unusableRMNodesConcurrentSet.add(eventNode);
            for(RMApp app: rmContext.getRMApps().values()) {
              this.rmContext
                  .getDispatcher()
                  .getEventHandler()
                  .handle(
                      new RMAppNodeUpdateEvent(app.getApplicationId(), eventNode,
                          RMAppNodeUpdateType.NODE_UNUSABLE));
            }
            break;
          case NODE_USABLE:
            if (unusableRMNodesConcurrentSet.contains(eventNode)) {
              LOG.debug(eventNode + " reported usable");
              unusableRMNodesConcurrentSet.remove(eventNode);
            }
            for (RMApp app : rmContext.getRMApps().values()) {
              this.rmContext
                  .getDispatcher()
                  .getEventHandler()
                  .handle(
                      new RMAppNodeUpdateEvent(app.getApplicationId(), eventNode,
                          RMAppNodeUpdateType.NODE_USABLE));
            }
            break;
          default:
            LOG.error("Ignoring invalid eventtype " + event.getType());
          }
        }
      

      Attachments

        1. 0001-YARN-3990.patch
          2 kB
          Bibin Chundatt
        2. 0002-YARN-3990.patch
          8 kB
          Bibin Chundatt
        3. 0003-YARN-3990.patch
          9 kB
          Bibin Chundatt

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            bibinchundatt Bibin Chundatt
            rohithsharma Rohith Sharma K S
            Votes:
            0 Vote for this issue
            Watchers:
            15 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment