[YARN-742] Log aggregation causes a lot of redundant setPermission calls

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.23.7, 2.0.4-alpha
• Fix Version/s: 2.1.0-beta, 0.23.9
• Component/s: nodemanager
• Target Version/s: 2.1.0-beta, 0.23.9
• Hadoop Flags: Reviewed
• Assignee: Jason Darrell Lowe
• Reporter: Kihwal Lee
• Created: 31/May/13 18:27
• Updated: 03/Sep/14 22:57
• Resolved: 05/Jun/13 17:43

Description

In one of our clusters, namenode RPC is spending 45% of its time on serving setPermission calls. Further investigation has revealed that most calls are redundantly made on /mapred/logs/<user>/logs. Also mkdirs calls are made before this.