Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-49569

Introduce Shim for missing spark/core classes

    XMLWordPrintableJSON

Details

    Description

      Introduce shims for SparkContext, RDD, and QueryExecution. This will have to be a in a separate module, and is supposed to be a compile time dependency for sql/api project, and an actual dependency for an independent Spark Connect Client.

      We need these three classes to support all user facing API in the sql/api project. This will allow us to make the classes the primary interface for Scala Dataset operations.

      For connect these methods will throw an (actionable) error, and for the classic client they will just work. On the connect side in the future we can use this to build better errors, and provide (method specific) mitigations. 

      Attachments

        Issue Links

          Activity

            People

              hvanhovell Herman van Hövell
              hvanhovell Herman van Hövell
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: