[SPARK-17264] DataStreamWriter should document that it only supports Parquet for now

• Type: Bug
• Status: Resolved
• Priority: Trivial
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.0.1, 2.1.0
• Component/s: Documentation, Input/Output

Environment

Mac OSX

• Assignee: Sean R. Owen
• Reporter: Bill Reed
• Created: 26/Aug/16 14:14
• Updated: 12/Dec/22 18:10
• Resolved: 30/Aug/16 10:20

Description

The API documentations for DataStreamWriter.format states "Specifies the underlying output data source. Built-in options include "parquet", "json", etc." but when specifying "json" or "text" for the format. the following exception is thrown:
Exception in thread "main" java.lang.UnsupportedOperationException: Data source json does not support streamed writing
at org.apache.spark.sql.execution.datasources.DataSource.createSink(DataSource.scala:273)
at org.apache.spark.sql.streaming.DataStreamWriter.start(DataStreamWriter.scala:291)

The only format that works is .format("parquet")

Issue Links

• is related to: SPARK-15472 Add support for writing partitioned `csv`, `json`, `text` formats in Structured Streaming
• links to: [Github] Pull Request #14860 (srowen)