Uploaded image for project: 'Livy'
  1. Livy
  2. LIVY-483

set spark memory configuration by SparkR code not working

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.5.0
    • Fix Version/s: None
    • Component/s: Core
    • Labels:
      None
    • Environment:
      Livy version: 0.5 version latest 2008 release
      Spark: 2.1.1
      JupyterNote book: 4.3.1
      SparkMagic: 0.11.3

      Description

      Hi,
       
      When we are trying to allocate spark memory by sparkR from jupyter notebook Spark always setting default config memory.We want to set spark memory configuration by SparkR code but with current Livy it's not working.Same things when trying with pySpark its working properly.I'm attaching both screenshots which we have tried from pySpark and sparkR with this mail. 

      If you need further any information regarding this issue please let us know.
       
      Livy version: 0.5 version latest 2008 release 
      Spark: 2.1.1
      JupyterNote book: 4.3.1 
      SparkMagic: 0.11.3

        Attachments

        1. Screenshot from 2018-07-12 10-53-41.png
          24 kB
          joyjit das
        2. Screenshot from 2018-07-12 10-53-12.png
          130 kB
          joyjit das
        3. Screenshot from 2018-07-12 10-54-41.png
          136 kB
          joyjit das
        4. Screenshot from 2018-07-12 10-57-04.png
          143 kB
          joyjit das
        5. Screenshot from 2018-07-12 10-57-25.png
          140 kB
          joyjit das
        6. Screenshot from 2018-07-12 10-54-59.png
          194 kB
          joyjit das

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              joyjit joyjit das
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: