Uploaded image for project: 'TOREE'
  1. TOREE
  2. TOREE-535

Spark init configurations (jars to driver and executors)

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Trivial
    • Resolution: Unresolved
    • None
    • 0.5.0
    • None
    • None

    Description

      Currently in v 0.5.0  there's no way to configure spark properties before spark initialization.

      Especially adding jars to spark driver and executors and then import the functions. `%Addjar` doesn't do that.

      Whereas with spylon-kernel one can invoke `%init_spark` to load the jars and it works fine when importing in other cells. e.g.

      %%init_spark
launcher.jars = ["/home/rgaspar/myjar-1.5.1.jar"]

      Attachments

        Issue Links

          is a clone of

          Improvement - An improvement or enhancement to an existing feature or task. TOREE-534 SQL magic - Render as table like dataframe magic

          • Trivial - Cosmetic problem like misspelt words or misaligned text.
          • Open

          Activity

            People

              Unassigned Unassigned
              RikG Ricardo Gaspar
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated: