[SPARK-18364] Expose metrics for YarnShuffleService

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.1
• Fix Version/s: 3.0.0
• Component/s: Spark Core, YARN
• Assignee: Marek Simunek
• Reporter: Steven Rand
• Created: 08/Nov/16 20:55
• Updated: 17/May/20 18:13
• Resolved: 01/Oct/18 16:05

Description

ExternalShuffleService exposes metrics as of SPARK-16405. However, YarnShuffleService does not.

The work of instrumenting ExternalShuffleBlockHandler was already done in SPARK-16405, so this JIRA is for creating a MetricsSystem in YarnShuffleService similarly to how ExternalShuffleService already does it.

Issue Links

• relates to: SPARK-16405 Add metrics and source for external shuffle service
• links to: [Github] Pull Request #17401 (ash211)
• links to: [Github] Pull Request #22485 (mareksimunek)
• links to: [Github] Pull Request #22485 (mareksimunek)
• links to: [Github] Pull Request #22498 (pgandhi999)