Description
I'm running spark on yarn and trying to use R in cluster mode. RRunner seems to just call Rscript and assumes its in the path. But on our YARN deployment R isn't installed on the nodes so it needs to be distributed along with the job and we need the ability to point to where it gets installed. sparkR in client mode has the config spark.sparkr.r.command to point to Rscript. RRunner should have something similar so it works in cluster mode
Attachments
Issue Links
- relates to
-
SPARK-11414 Forgot to update usage of "spark.sparkr.r.command" in RRDD in the PR for SPARK-10971
-
- Resolved
-
- links to
