[SPARK-30097] Adding support for core writers

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Won't Do
• Affects Version/s: 3.0.0
• Component/s: SQL, Structured Streaming

Environment

 

• Assignee: Unassigned
• Reporter: German Schiavon Matteo
• Created: 02/Dec/19 07:51
• Updated: 01/Feb/20 23:42
• Resolved: 01/Feb/20 23:42

Description

When using writeStream we always have to use format("xxx") in order to target the selected sink while in r*eadStream* you can use directly .parquet 

Basically this is to add the support to the core writers for writeStream

Example:
 

writeStream
.outputMode("append")
.partitionBy("id")
.options(options)
.parquet(path)
 

 

Issue Links

• links to: GitHub Pull Request #26733