[SPARK-3797] Run the shuffle service inside the YARN NodeManager as an AuxiliaryService

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.1.0
• Fix Version/s: 1.2.0
• Component/s: YARN
• Target Version/s: 1.2.0
• Assignee: Andrew Or
• Reporter: Patrick Wendell
• Created: 04/Oct/14 19:08
• Updated: 06/Nov/14 22:29
• Resolved: 05/Nov/14 23:43

Description

It's also worth considering running the shuffle service in a YARN container beside the executor(s) on each node.

Issue Links

• depends upon: SPARK-3796 Create shuffle service for external block storage
• links to: [Github] Pull Request #3082 (andrewor14)
• links to: [Github] Pull Request #3144 (andrewor14)