[SPARK-31121] catalog plugin API

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0
• Component/s: SQL
• Assignee: Ryan Blue
• Reporter: Wenchen Fan
• Created: 11/Mar/20 11:35
• Updated: 10/Apr/20 13:32
• Resolved: 11/Mar/20 11:39

Description

Details please see the SPIP doc: https://docs.google.com/document/d/1zLFiA1VuaWeVxeTDXNg8bL6GP3BVoOZBkewFtEnjEoo/edit#heading=h.m45webtwxf2d

This will bring multi-catalog support to Spark and allow external catalog implementations.

Sub-Tasks

• DataSourceV2: Add catalog support: Resolved
• Add SupportsNamespaces interface for v2 catalogs: Resolved
• create a public API for V2SessionCatalog: Resolved
• centralize the catalog and table lookup logic: Resolved
• DataSourceV2: Support CREATE NAMESPACE: Resolved
• DataSourceV2: Support DROP NAMESPACE: Resolved
• DataSourceV2: Support SHOW DATABASES: Resolved
• DataSourceV2: Support SHOW CURRENT NAMESPACE: Resolved
• DataSourceV2: Add CURRENT_CATALOG function: Resolved
• DataSourceV2: Support ALTER TABLE statements in catalyst SQL parser: Resolved
• DataSourceV2: Support SHOW TABLES: Resolved
• DataSourceV2: Add AlterTable v2 implementation: Closed