[FLINK-10556] Integration with Apache Hive

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.6.0
• Fix Version/s: 1.10.0
• Component/s: Connectors / Hive, Table SQL / Ecosystem
• Assignee: Xuefu Zhang
• Reporter: Xuefu Zhang
• Created: 15/Oct/18 19:03
• Updated: 03/Dec/19 05:26
• Resolved: 03/Dec/19 05:26

Description

This is an umbrella JIRA tracking all enhancement and issues related to integrating Flink with Hive ecosystem. This is an outcome of a discussion in the community, and thanks go to everyone that provided feedback and interest.

Specifically, we'd like to see the following features and capabilities immediately in Flink:

1. Metadata interoperability
2. Data interoperability
3. Data type compatibility
4. Hive UDF support
5. DDL/DML/Query language compatibility

For a longer term, we'd also like to add or improve:

1. Compatible SQL service, client tools, JDBC/ODBC drivers
2. Better task failure tolerance and task scheduling
3. Support other user customizations in Hive (storage handlers, serdes, etc).

I will provide more details regarding the proposal in a doc shortly. Design doc, if deemed necessary, will be provided in each related sub tasks under this JIRA.

Feedback and contributions are greatly welcome!

Issue Links

• is a parent of: FLINK-10729 Create a Hive connector for Hive data access in Flink
• is a parent of: FLINK-10744 Integrate Flink with Hive metastore
• relates to: FLINK-10232 Add a SQL DDL
• relates to: FLINK-10686 Introduce a flink-table-common module

Sub-Tasks

• Flink SQL calling Hive User-Defined Functions: Closed
• Support for simple hive UDF: Closed
• Support for Hive GenericUDF: Closed
• Support for Hive User-defined Table-generating Function (UDTF): Closed
• Support for Hive's User-defined Aggregation Function (UDAF): Closed
• Add user documentation for Flink-Hive integration: Closed