[SPARK-42522] Fix DataFrameWriterV2 to find the default source

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: Connect
• Assignee: Takuya Ueshin
• Reporter: Takuya Ueshin
• Created: 21/Feb/23 21:42
• Updated: 22/Feb/23 20:53
• Resolved: 22/Feb/23 20:53

Description

df.writeTo("test_table").create()

throws:

pyspark.errors.exceptions.connect.SparkConnectGrpcException: (org.apache.spark.SparkClassNotFoundException) [DATA_SOURCE_NOT_FOUND] Failed to find the data source: . Please find packages at `https://spark.apache.org/third-party-projects.html`.

Issue Links

• links to: [Github] Pull Request #40109 (ueshin)