[FLINK-11275] Unified Catalog APIs

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Table SQL / API
• Assignee: Xuefu Zhang
• Reporter: Xuefu Zhang
• Created: 07/Jan/19 23:52
• Updated: 05/Jun/19 21:52

Description

During Flink-Hive integration, we found that the current Catalog APIs are quite cumbersome and to a large extent requiring significant rework. While previous APIs are essentially not used, at least not in Flink code base, we needs to be careful in defining a new set of APIs to avoid future rework again.

This is an uber JIRA covering all the work outlined in FLIP-30.

Issue Links

• contains: FLINK-12753 Implement GenericInMemoryCatalog
• contains: FLINK-12755 implement HiveCatalog
• contains: FLINK-12256 Implement KafkaReadableCatalog
• contains: FLINK-12278 Support user implemented and customized catalogs thru dynamic class loading
• is related to: FLINK-12625 Support catalogs in SQL Client

Sub-Tasks

• Handle existing table registration via YAML file in the context of catalog support: Open
• Integrate HiveTableFactory with existing table factory discovery mechanism: Open
• Rename a few exception class names that were migrated from scala: Open
• Add more detailed javadoc for getDescription() and getDetailedDescription() in catalog object interfaces: In Progress
• Support partitioned view in catalog API: Open
• Add column stats for decimal type: Open
• Add some more (negative) test cases for FLINK-12365: Open
• Convert CatalogView to org.apache.calcite.schema.Table so that planner can use unified catalog APIs: Open
• add Chinese documentation for catalogs: Open
• create CatalogTableBuilder to support building CatalogTable from descriptors: Open