Details
-
New Feature
-
Status: Closed
-
Major
-
Resolution: Invalid
-
1.13.1
-
None
-
None
-
flink: 1.13.1
flink calcite: 1.26.1
kafka-eagle: 2.0.5
kafka-eagle calcite 1.21.0
Description
Hello everyone
At present, there are codes that depend on hive in many places of the parsed functions and tables. The functions and tables of hive in individual dialects need to be parsed according to the part of flink, so try the analysis method based on flink SPI, which is compatible with kafka's DDL DML
I observe the community’s proposal
Flip-152: https://cwiki.apache.org/confluence/display/FLINK/FLIP-152%3A+Hive+Query+Syntax+Compatibility
We can know that
```
CREATE CATALOG myhive WITH (
'Type' ='hive',
'Default-database' ='default',
'Hive-conf-dir' ='/data/hive/conf/'
);
To
USE CATALOG myhive;
Set table.sql-dialect=hive;
```
When specifying the sql-dialect type, it will follow the SPI from the configuration file
The org.apache.flink.table.factories.TableFactory file under flink-connector-hive is assigned to HiveParserFactory to parse the corresponding grammar, and HiveParser is the corresponding grammar parser
And perform grammatical analysis in HiveParserFactory#create
During the course of investigating kafka-eagle, I found
Among them, KSQL is also based on calcite for grammatical analysis, and can support DDL and DML of kafka tables
Test-related classes are in: KSqlParser#TestKSqlParser
And completed the analysis of the corresponding grammar in KsqlParser#parseQueryKSql
Does the community have any good suggestions for this proposal?