Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-15777

Catalog federation

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • None
    • None
    • SQL

    Description

      This is a ticket to track progress to support federating multiple external catalogs. This would require establishing an API (similar to the current ExternalCatalog API) for getting information about external catalogs, and ability to convert a table into a data source table.

      As part of this, we would also need to be able to support more than a two-level table identifier (database.table). At the very least we would need a three level identifier for tables (catalog.database.table). A possibly direction is to support arbitrary level hierarchical namespaces similar to file systems.

      Once we have this implemented, we can convert the current Hive catalog implementation into an external catalog that is "mounted" into an internal catalog.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              rxin Reynold Xin
              Votes:
              7 Vote for this issue
              Watchers:
              38 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: