[FLINK-26603] [Umbrella] Decouple Hive with Flink planner

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.18.0
• Component/s: Connectors / Hive, Table SQL / Planner

Release Note

Users can use the Hive dialect directly without the extra jar swapping. Meanwhile, if you have put flink-sql-connector-hive-3.1.3.jar in FLINK_HOME/lib, please make sure flink-table-planner jar is not in FLINK_HOME/lib, otherwise it'll throw exception, more detail can be seen in FLINK-33168. Please also note to make Hive dialect more clear, Flink community has decided not to fall back to Flink's default dialect implicitly when it's in Hive dialect. So, it may cause some queries specified to Flink to fail in Hive dialect. In that case, please switch to Flink dialect explicitly. If you find some statements specified to Flink also should be included in Hive dialect for convenience, please report it to Flink community.

• Assignee: luoyuxia
• Reporter: luoyuxia
• Created: 11/Mar/22 09:25
• Updated: 25/Oct/23 02:58
• Resolved: 19/Oct/23 13:04

Description

To support Hive dialect with Flink, we have implemented FLIP-123, FLIP-152.

But it also brings much maintenance burden and complexity for it mixes some logic specific to Hive with Flink planner. We should remove such logic from Flink planner and make it totally decouple with Flink planner.

With this ticket, we expect:

1:  there won't be any specific logic to Hive in planner module

2:  remove  flink-sql-parser-hive from flink-table module 

3:  remove the planner dependency in flink-connector-hive

I'll update more details after investigation.

 

Issue Links

• blocks: FLINK-30064 Move existing Hive connector code from Flink repo to dedicated Hive repo
• blocks: FLINK-30063 Create and initialize repository for Hive connector
• is related to: FLINK-26360 [Umbrella] Improvement for Hive Query Syntax Compatibility

Sub-Tasks

• remove flink-sql-parser-hive dependency in table-planner: Resolved
• drop flink-sql-parser-hive: Closed
• move SQLClientHiveITCase and TestHiveCatalogFactory to flink-connector-hive e2e: Resolved
• [Connector/Hive] cleanup hive/haoop package ambiguous package dependencies: Closed
• remove the planner @internal dependency in flink-connector-hive: Resolved
• Hive dialect should use public interfaces in Hive connector: Resolved
• Remove provided dependency of flink-table-planner from Hive connector: Closed
• Don't swap table-planner-loader and table-planner to use hive dialect: Resolved