[SPARK-47492] Relax definition of whitespace in lexer

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0
• Component/s: Spark Core
• Labels: pull-request-available
• Assignee: Serge Rielau
• Reporter: Serge Rielau
• Shepherd: Wenchen Fan
• Created: 20/Mar/24 19:23
• Updated: 28/Mar/24 22:51
• Resolved: 28/Mar/24 22:51

Description

There have been multiple incidences where queries "copied" in from other sources resulted in "weird" syntax errors which ultimately boiled down to whitespaces which the lexer does not recognize as such.

Issue Links

• links to: GitHub Pull Request #45620