Details
-
New Feature
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
4.0.0
Description
Introduce shims for SparkContext, RDD, and QueryExecution. This will have to be a in a separate module, and is supposed to be a compile time dependency for sql/api project, and an actual dependency for an independent Spark Connect Client.
We need these three classes to support all user facing API in the sql/api project. This will allow us to make the classes the primary interface for Scala Dataset operations.
For connect these methods will throw an (actionable) error, and for the classic client they will just work. On the connect side in the future we can use this to build better errors, and provide (method specific) mitigations.
Attachments
Issue Links
- blocks
-
SPARK-49700 Switch interface and implementations
-
- In Progress
-
- links to