Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-3124 Add external metadata notification mechanisms to Impala
  3. IMPALA-3681

Automatic invalidation of HMS metadata using Hive replication API (HIVE-7973)

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: Impala 2.6.0
    • Fix Version/s: None
    • Component/s: Catalog
    • Labels:

      Description

      Hive is adding support for a replication API (https://issues.apache.org/jira/browse/HIVE-7973) that allows an HMS client to retrieve a global stream of HMS modifications made by all the HMS processes (in case of HMS HA). The Catalog server could use that API to automatically collect and update cached HMS metadata, thus eliminating the need for the user to call invalidate and refresh. This can only handle HMS metadata; file and block metadata will still need to be updated from the Namenode (e.g. by using inotify).

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                dtsirogiannis Dimitris Tsirogiannis
              • Votes:
                2 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: