[SPARK-30049] SQL fails to parse when comment contains an unmatched quote character

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0, 3.1.0
• Component/s: SQL
• Target Version/s: 3.0.0
• Assignee: Javier Fuentes
• Reporter: Jason Darrell Lowe
• Created: 26/Nov/19 20:49
• Updated: 12/Dec/22 18:11
• Resolved: 03/Mar/20 15:57

Description

A SQL statement that contains a comment with an unmatched quote character can lead to a parse error. These queries parsed correctly in older versions of Spark. For example, here's an excerpt from an interactive spark-sql session on a recent Spark-3.0.0-SNAPSHOT build (commit e23c135e568d4401a5659bc1b5ae8fc8bf147693):

spark-sql> SELECT 1 -- someone's comment here
         > ;
Error in query: 
extraneous input ';' expecting <EOF>(line 2, pos 0)

== SQL ==
SELECT 1 -- someone's comment here
;
^^^

Issue Links

• causes: SPARK-31102 spark-sql fails to parse when contains comment
• is related to: SPARK-26321 Split a SQL in a correct way
• links to: GitHub Pull Request #27321