Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-8506

SparkR does not provide an easy way to depend on Spark Packages when performing init from inside of R

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 1.4.0
    • 1.4.1, 1.5.0
    • SparkR
    • None

    Description

      While packages can be specified when using the sparkR or sparkSubmit scripts, the programming guide tells people to create their spark context using the R shell + init. The init does have a parameter for jars but no parameter for packages. Setting the SPARKR_SUBMIT_ARGS overwrites some necessary information. I think a good solution would just be adding another field to the init function to allow people to specify packages in the same way as jars.

      Attachments

        Activity

          People

            holden Holden Karau
            holden Holden Karau
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: