[HIVE-21059] Support external catalogs

• Type: New Feature
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Thejas Nair
• Created: 20/Dec/18 01:01
• Updated: 20/Dec/18 17:20

Description

Hive has ability to query data from external sources such as other RDBMS, Kafka, Druid, Hbase.

For example, to be able to query data from external sources such as a mysql table, an external table has to be explicitly created in Hive for every table in mysql that needs to be made accessible.
Moreover, for creating such a table, the schema and login credentials have to be specified.

By supporting "external catalogs" in Hive, we can have references to all tables in an entire mysql database by just creating one external catalog. The schema of the tables would also get automatically detected from the underlying source.

Where possible, additional information such as statistics of the tables can also be imported from the underlying datasource, to enable Hive cost based optimizer to create optimized query plans.

To be able to support the use of external catalog, some of the work tracked under HIVE-18685 for catalog support (including catalog in SQL syntax of Hive) is also needed.

Issue Links

• is blocked by: HIVE-18685 Add catalogs to Hive
• is related to: HIVE-21060 JDBCStorageHandler should auto discover external schema