[SPARK-14062] Put metrics.properties to distributed cache for Spark running on Yarn. Also Fix log4j propagation issue

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Component/s: Spark Core, YARN
• Assignee: Saisai Shao
• Reporter: Saisai Shao
• Created: 22/Mar/16 07:11
• Updated: 17/May/20 18:17
• Resolved: 31/Mar/16 17:28

Description

Currently for Spark running on Yarn, if we want to specify metrics configurations for executors, we have to manually upload property file into executors with --files. Actually for spark running on yarn, this could be done by adding to distributed cache implicitly, like "log4j.properties". Here proposed to add this improvement for "metrics.properties".

Issue Links

• links to: [Github] Pull Request #11885 (jerryshao)