[SPARK-21934] Expose Netty memory usage via Metrics System

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.3.0
• Fix Version/s: 2.3.0
• Component/s: Spark Core
• Assignee: Saisai Shao
• Reporter: Saisai Shao
• Created: 06/Sep/17 08:37
• Updated: 21/Sep/17 05:57
• Resolved: 21/Sep/17 05:56

Description

This is a follow-up work of SPARK-9104 to expose the Netty memory usage to MetricsSystem. My initial thought is to only expose Shuffle memory usage, since shuffle is a major part of memory usage in network communication compared to RPC, file server, block transfer.

If user wants to also expose Netty memory usage for other modules, we could add more metrics later.

Issue Links

• links to: [Github] Pull Request #19160 (jerryshao)