[FLINK-10281] Fix string literal escaping throughout Table & SQL API

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.6.0
• Fix Version/s: 1.7.0
• Component/s: Table SQL / API
• Labels: pull-request-available
• Assignee: Timo Walther
• Reporter: vinoyang
• Created: 05/Sep/18 03:30
• Updated: 12/Sep/18 14:20
• Resolved: 12/Sep/18 14:19

Description

for example,  regular expression matches text ("\w") or number ("\d") :

testAllApis(
  "foothebar".regexExtract("foo([\\w]+)", 1),       //OK, the method got 'foo([\w]+)'
  "'foothebar'.regexExtract('foo([\\\\w]+)', 1)",   //failed, the method got 'foo([\\w]+)' returns "", but if pass 'foo([\\w]+)' would get compile error.
  "REGEX_EXTRACT('foothebar', 'foo([\\\\w]+)', 1)", //OK, the method got 'foo([\w]+)' but must pass four '\'
  "thebar"
)

the "similar to" function has the same issue.

 

Update:

Proper escaping of quotes was not possible in the past for Table API. SQL and SQL Client
were not standard compliant.

Due to FLINK-8301 backslashes were considered in SQL literals, however, they
should only be used in SQL `U&'\1234'` literals. For the Table API, the new
logic relies on the Java/Scala escaping and uses duplicate quotes for escaping
the quotes in expression strings. For SQL, we rely on unicode string literals
with or without the UESCAPE clause. The SQL Client was using backslashes for
escaping new lines. For the SQL Client, we allow unescaped new lines and
use ';' for statement finalization; similar to other SQL clients.

 

Issue Links

• links to: GitHub Pull Request #6659
• links to: GitHub Pull Request #6671