Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-42561

Add TempView APIs to Dataset

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.4.0
    • 3.4.1
    • Connect
    • None

    Description

      Add all temp view APIs to connect's Dataset.

      /**
      * Registers this Dataset as a temporary table using the given name. The lifetime of this
      * temporary table is tied to the [[SparkSession]] that was used to create this Dataset.
      *
      * @group basic
      * @since 1.6.0
      */
      @deprecated("Use createOrReplaceTempView(viewName) instead.", "2.0.0")
      def registerTempTable(tableName: String): Unit
      
      /**
      * Creates a local temporary view using the given name. The lifetime of this
      * temporary view is tied to the [[SparkSession]] that was used to create this Dataset.
      *
      * Local temporary view is session-scoped. Its lifetime is the lifetime of the session that
      * created it, i.e. it will be automatically dropped when the session terminates. It's not
      * tied to any databases, i.e. we can't use `db1.view1` to reference a local temporary view.
      *
      * @throws AnalysisException if the view name is invalid or already exists
      *
      * @group basic
      * @since 2.0.0
      */
      @throws[AnalysisException]
      def createTempView(viewName: String): Unit
      
      /**
      * Creates a local temporary view using the given name. The lifetime of this
      * temporary view is tied to the [[SparkSession]] that was used to create this Dataset.
      *
      * @group basic
      * @since 2.0.0
      */
      def createOrReplaceTempView(viewName: String): Unit
      
      /**
      * Creates a global temporary view using the given name. The lifetime of this
      * temporary view is tied to this Spark application.
      *
      * Global temporary view is cross-session. Its lifetime is the lifetime of the Spark application,
      * i.e. it will be automatically dropped when the application terminates. It's tied to a system
      * preserved database `global_temp`, and we must use the qualified name to refer a global temp
      * view, e.g. `SELECT * FROM global_temp.view1`.
      *
      * @throws AnalysisException if the view name is invalid or already exists
      *
      * @group basic
      * @since 2.1.0
      */
      @throws[AnalysisException]
      def createGlobalTempView(viewName: String): Unit
      
      /**
      * Creates or replaces a global temporary view using the given name. The lifetime of this
      * temporary view is tied to this Spark application.
      *
      * Global temporary view is cross-session. Its lifetime is the lifetime of the Spark application,
      * i.e. it will be automatically dropped when the application terminates. It's tied to a system
      * preserved database `global_temp`, and we must use the qualified name to refer a global temp
      * view, e.g. `SELECT * FROM global_temp.view1`.
      *
      * @group basic
      * @since 2.2.0
      */
      def createOrReplaceGlobalTempView(viewName: String): Unit 

      Attachments

        Activity

          People

            amaliujia Rui Wang
            hvanhovell Herman van Hövell
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: