[SPARK-19522] --executor-memory flag doesn't work in local-cluster mode

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 2.0.0
• Component/s: Deploy
• Labels: bulk-closed
• Assignee: Andrew Or
• Reporter: Andrew Or
• Created: 08/Feb/17 22:43
• Updated: 21/May/19 04:11
• Resolved: 21/May/19 04:11

Description

bin/spark-shell --master local-cluster[2,1,2048]

doesn't do what you think it does. You'll end up getting executors with 1GB each because that's the default. This is because the executor memory flag isn't actually read in local mode.

Issue Links

• links to: [Github] Pull Request #16975 (andrewor14)