[YARN-10784] Optimize RMNodeStatusEvent process performance

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.8.5
• Fix Version/s: 3.3.0
• Assignee: chaosju
• Reporter: chaosju
• Created: 24/May/21 11:31
• Updated: 24/May/21 12:04
• Resolved: 24/May/21 12:04

Description

According to YARN-9927 analysis. The RMNodeStatusEvent dominates 90% time consumption of RM event scheduler.

Use async-profiler profile the Processing of RMNodeStatusEvent，ProtoUtils.convertFromProtoFormat Use JDK String.replace().

It is definitely worth considering not using these JDK String methods and opting for the faster Apache Commons StringUtils.replace() instead.

Test case on jdk8:

for (int i=0 ; i<100000 ;i++)

{ ProtoUtils.convertFromProtoFormat(builder.getState()); }

Apache Commons StringUtils.replace() : used 30ms
JDK String.replace() : used 60ms

Issue Links

• Blocked: YARN-10696 Add RMNodeEvent to single async dispatcher before YARN-9927.
• Blocked: YARN-9927 RM multi-thread event processing mechanism
• is duplicated by: YARN-9624 Use switch case for ProtoUtils#convertFromProtoFormat containerState