[YARN-9640] Slow event processing could cause too many attempt unregister events

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 3.3.0, 3.2.1
• Labels: scalability
• Target Version/s: 3.2.1
• Hadoop Flags: Reviewed
• Assignee: Bibin Chundatt
• Reporter: Bibin Chundatt
• Created: 22/Jun/19 01:39
• Updated: 29/Aug/19 04:08
• Resolved: 29/Aug/19 04:08

Description

We found in one of our test cluster verification that the number attempt unregister events is about 300k+.

1. AM all containers completed.
2. AMRMClientImpl send finishApplcationMaster
3. AMRMClient check event 100ms the finish Status using finishApplicationMaster request.
4. AMRMClientImpl#unregisterApplicationMaster

         while (true) {
           FinishApplicationMasterResponse response =
               rmClient.finishApplicationMaster(request);
           if (response.getIsUnregistered()) {
             break;
           }
           LOG.info("Waiting for application to be successfully unregistered.");
           Thread.sleep(100);
         }
   

1. ApplicationMasterService finishApplicationMaster interface sends unregister events on every status update.

We should send unregister event only once and cache event send , ignore and send not unregistered response back to AM not overloading the event queue.