[FLINK-15644] Add support for SQL query validation

• Type: New Feature
• Status: Open
• Priority: Not a Priority
• Resolution: Unresolved
• Component/s: Table SQL / API
• Labels: auto-deprioritized-major auto-deprioritized-minor
• Assignee: Unassigned
• Reporter: Fabian Hueske
• Created: 17/Jan/20 17:31
• Updated: 19/Nov/21 22:38

Description

It would be good if the TableEnvironment would offer methods to check the validity of SQL queries. Such a method could be used by services (CLI query shells, notebooks, SQL UIs) that are backed by Flink and execute their queries on Flink.

Validation should be available in two levels:

1. Validation of syntax and semantics: This includes parsing the query, checking the catalog for dbs, tables, fields, type checks for expressions and functions, etc. This will check if the query is a valid SQL query.
2. Validation that query is supported: Checks if Flink can execute the given query. Some syntactically and semantically valid SQL queries are not supported, esp. in a streaming context. This requires running the optimizer. If the optimizer generates an execution plan, the query can be executed. This check includes the first step and is more expensive.

The reason for this separation is that the first check can be done much fast as it does not involve calling the optimizer. Hence, it would be suitable for fast checks in an interactive query editor. The second check might take more time (depending on the complexity of the query) and might not be suitable for rapid checks but only on explicit user request.

Requirements:

• validation does not modify the state of the TableEnvironment, i.e. it does not add plan operators
• validation does not require connector dependencies
• validation can identify the update mode of a continuous query result (append-only, upsert, retraction).

Out of scope for this issue:

• better error messages for unsupported features as suggested by FLINK-7217