Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-19540

Add ability to clone SparkSession with an identical copy of the SessionState

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.2.0
    • 2.2.0
    • SQL
    • None

    Description

      Forking a newSession() from SparkSession currently makes a new SparkSession that does not retain SessionState (i.e. temporary tables, SQL config, registered functions etc.) This change adds a method cloneSession() which creates a new SparkSession with a copy of the parent's SessionState.
      Subsequent changes to base session are not propagated to cloned session, clone is independent after creation.
      If the base is changed after clone has been created, say user registers new UDF, then the new UDF will not be available inside the clone. Same goes for configs and temp tables.

      Attachments

        Issue Links

          Activity

            People

              kunalkhamar Kunal Khamar
              kunalkhamar Kunal Khamar
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: