Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-3124 Add external metadata notification mechanisms to Impala
  3. IMPALA-3681

Automatic invalidation of HMS metadata using Hive replication API (HIVE-7973)

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Minor
    • Resolution: Duplicate
    • Impala 2.6.0
    • None
    • Catalog

    Description

      Hive is adding support for a replication API (https://issues.apache.org/jira/browse/HIVE-7973) that allows an HMS client to retrieve a global stream of HMS modifications made by all the HMS processes (in case of HMS HA). The Catalog server could use that API to automatically collect and update cached HMS metadata, thus eliminating the need for the user to call invalidate and refresh. This can only handle HMS metadata; file and block metadata will still need to be updated from the Namenode (e.g. by using inotify).

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              dtsirogiannis Dimitris Tsirogiannis
              Votes:
              2 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: