[SPARK-3674] Add support for launching YARN clusters in spark-ec2

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.4.0
• Component/s: EC2
• Target Version/s: 1.4.0
• Assignee: Shivaram Venkataraman
• Reporter: Shivaram Venkataraman
• Created: 24/Sep/14 03:33
• Updated: 27/May/15 01:56
• Resolved: 26/May/15 22:02

Description

Right now spark-ec2 only supports launching Spark Standalone clusters. While this is sufficient for basic usage it is hard to test features or do performance benchmarking on YARN. It will be good to add support for installing, configuring a Apache YARN cluster at a fixed version – say the latest stable version 2.4.0

Issue Links

• is related to: SPARK-5971 Add Mesos support to spark-ec2
• links to: [Github] Pull Request #6376 (shivaram)
• links to: [Github] Pull Request #6424 (shivaram)