Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-9670

Fix unloaded views are shown as tables for GET_TABLES requests

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Catalog
    • Labels:
      None

      Description

      JDBC/ODBC clients will send GET_TABLES requests to get the list of tables of a given db. Table types and comments should be returned as well. However, for unloaded views which are represended as IncompleteTable, Impala doesn't know the exact table types and return them as tables by default. This causes some troubles for applications that needs to distinguish views.

      Problems to fix in this JIRA:

      • Views are shown as tables when Impala launches (all views are unloaded).
      • Views are shown as tables after creation (the created view is unloaded).

      Possible solution is adding table type to IncompleteTable:

      • _IncompleteTable_s are created in several places. Add table type in those places.
      • For startup and global invalidate metadata,
        • Approach 1: use getTables API with table type.
          • currently we use IMetaStoreClient#getAllTables(dbName) to get table names inside a db and create IncompleteTables for them. 
          • We can add an additional call to load names of views using IMetaStoreClient#getTables(dbName, tablePattern, tableType).
          • This may increase the startup time by 2x.
        • Approach 2: use getTableMeta API instead of getAllTables.
          • Pros: can get the comments as well. 
          • Cons: This may increase the startup time by 4x.
      • Load view's metadata at the end of lightweight DDL operations like "INVALIDATE METADATA table", "CREATE VIEW", etc.
      • EventProcessors can extract table type from the event's HMS table object.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                stigahuang Quanlong Huang
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated: