[SPARK-9092] Make --num-executors compatible with dynamic allocation

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.2.0
• Fix Version/s: 1.5.0
• Component/s: YARN
• Assignee: Niranjan Padmanabhan
• Reporter: Niranjan Padmanabhan
• Created: 16/Jul/15 02:52
• Updated: 13/Aug/15 08:59
• Resolved: 12/Aug/15 23:13

Description

Currently when you enable dynamic allocation, you can't use --num-executors or the property spark.executor.instances. If we are to enable dynamic allocation by default, we should make these work so that existing workloads don't fail

Issue Links

• links to: [Github] Pull Request #7657 (neurons)