[SPARK-22292] Add spark.mem.max to limit the amount of memory received from Mesos

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 2.2.0
• Component/s: Mesos
• Labels: bulk-closed

Environment

Spark with Mesos

• Assignee: Unassigned
• Reporter: windkithk
• Created: 17/Oct/17 05:33
• Updated: 21/May/19 04:16
• Resolved: 21/May/19 04:16

Description

To limit the amount of resources a spark job accept from Mesos, currently we can only use `spark.cores.max` to limit in terms of cpu cores.
However, when we have big memory executors, it would consume all the resources.

Issue Links

• links to: [Github] Pull Request #19510 (windkit)