Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Incomplete
-
2.2.0
-
None
Description
Hi,
If we have a dataframe with the column value as
ab\,cd\,ef\,gh
Then while writing it is being written as
"ab\,cd\\,ef\\,gh"
i.e it double escapes all the already escaped commas/delimiters but not the first one.
This is weird behaviour considering either it should do for all or none.
If I do mention df.option("escape","") as empty then it solves this problem but the double quotes inside the same value if any are preceded by a special char i.e '\u00'. Why does it do so when the escape character is set as ""(empty)?
Attachments
Attachments
Issue Links
- relates to
-
SPARK-21678 Disabling quotes while writing a dataframe
- Closed