[YARN-6339] Improve performance for createAndGetApplicationReport

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.9.0, 3.0.0-alpha4, 2.8.2
• Hadoop Flags: Reviewed
• Assignee: yunjiong zhao
• Reporter: yunjiong zhao
• Created: 14/Mar/17 23:03
• Updated: 07/Jun/17 21:40
• Resolved: 27/Mar/17 21:07

Description

There are two performance issue when calling createAndGetApplicationReport:
One is inside ProtoUtils.convertFromProtoFormat, replace is too slow for clusters which have more than 3000 nodes. Use substring is much better: https://issues.apache.org/jira/browse/YARN-6285?focusedCommentId=15923241&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15923241

Another one is inside getLogAggregationReportsForApp, if some application's LogAggregationStatus is TIME_OUT, every time it was called it will create an HashMap which will produce lots of garbage.

Issue Links

• is related to: YARN-6285 Add option to set max limit on ResourceManager for ApplicationClientProtocol.getApplications