[SPARK-17605] Add option spark.usePython and spark.useR for applications that use both pyspark and sparkr

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Component/s: Spark Submit
• Assignee: Unassigned
• Reporter: Jeff Zhang
• Created: 20/Sep/16 08:19
• Updated: 17/Feb/17 09:12
• Resolved: 17/Feb/17 09:12

Description

For applications (zeppelin & livy ) that use both pyspark and sparkr, we have to duplicate the code in SparkSubmit to figure the where is the requirement resources for pyspark & sparkr (pyspark.zip, py4j, sparkr.zip and R file in jars). It would be better to provide option spark.usePython and spark.useR, so that downstream project which use both pyspark and sparkr in one spark application don't need to duplicate the code in SparkSubmit, can just leverage these 2 options.

Issue Links

• links to: [Github] Pull Request #15159 (zjffdu)