[SPARK-25373] Support mixed language pipelines on Spark on K8s

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 3.1.0
• Component/s: Kubernetes, PySpark, R, Spark Core
• Assignee: Unassigned
• Reporter: Holden Karau
• Created: 08/Sep/18 00:10
• Updated: 17/May/20 18:25

Description

It would be good if we supported having Python/R workers in Spark on K8s even if it's not the language used to launch the driver program. An example of this is sparklingml, and another more common one would be something like livy.