[SPARK-35250] SQL DataFrameReader unescapedQuoteHandling parameter is misdocumented

• Type: Documentation
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.2
• Fix Version/s: 3.1.2, 3.2.0
• Component/s: Documentation
• Labels: starter

Environment

 

• Assignee: Hyukjin Kwon
• Reporter: Tim McNamara
• Created: 28/Apr/21 01:39
• Updated: 12/Dec/22 18:10
• Resolved: 03/May/21 23:50

Description

The unescapedQuoteHandling parameter of DataFrameReader isn't correctly documented. STOP_AT_DELIMITER appears twice, and it looks like that's overwritten an intended option, e.g. https://github.com/apache/spark/blob/1d550c4e90275ab418b9161925049239227f3dc9/sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala#L744-L749

To view instances where this error occurs, this is a useful query https://github.com/apache/spark/search?q=STOP_AT_DELIMITER

It appears that this bug was introduced here: https://github.com/apache/spark/pull/30518

Issue Links

• links to: [Github] Pull Request #32423 (HyukjinKwon)