[SPARK-24765] Add custom Kubernetes scheduler config parameter to spark-submit

• Type: New Feature
• Status: Resolved
• Priority: Minor
• Resolution: Information Provided
• Affects Version/s: 2.3.1
• Component/s: Kubernetes, Spark Core
• Assignee: Unassigned
• Reporter: Nihal Harish
• Created: 09/Jul/18 23:45
• Updated: 17/May/20 18:24
• Resolved: 10/Jul/18 18:13

Description

spark submit currently does not accept any config parameter that can enable the driver and executor pods to be scheduled by a custom scheduler as opposed to just the default-scheduler.

I propose the addition of a new config parameter:

spark.kubernetes.schedulerName