Description
SparkR currently does not work in YARN cluster mode as the R package is not shipped along with the assembly jar to the YARN AM. We could try to use the support for archives in YARN to send out the R package as a zip file.
Attachments
Attachments
Issue Links
- is related to
-
SPARK-6869 Add pyspark archives path to PYTHONPATH
- Resolved
-
SPARK-10500 sparkr.zip cannot be created if $SPARK_HOME/R/lib is unwritable
- Resolved
- relates to
-
SPARK-11524 Support SparkR with Mesos cluster
- Resolved
-
SPARK-8041 Consistently pass SparkR library directory to SparkR application
- Closed
- links to