[SPARK-22835] Spark toJSON gives wrong results when there is quote sign in value

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Invalid
• Affects Version/s: 2.1.1
• Component/s: SQL
• Assignee: Unassigned
• Reporter: Maciej Bryński
• Created: 19/Dec/17 21:50
• Updated: 19/Dec/17 21:53
• Resolved: 19/Dec/17 21:53

Description

When there is a quote in field value toJSON method gives wrong results.

>>> spark.range(1).selectExpr('concat("\\"", id, "\\"") as c').toJSON().collect()
['{"c":"\\"0\\""}']

The result is not a valid json.
https://jsonlint.com/