Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-3124 Add external metadata notification mechanisms to Impala
  3. IMPALA-3681

Automatic invalidation of HMS metadata using Hive replication API (HIVE-7973)

Attach filesAttach ScreenshotVotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: Impala 2.6.0
    • Fix Version/s: None
    • Component/s: Catalog
    • Labels:

      Description

      Hive is adding support for a replication API (https://issues.apache.org/jira/browse/HIVE-7973) that allows an HMS client to retrieve a global stream of HMS modifications made by all the HMS processes (in case of HMS HA). The Catalog server could use that API to automatically collect and update cached HMS metadata, thus eliminating the need for the user to call invalidate and refresh. This can only handle HMS metadata; file and block metadata will still need to be updated from the Namenode (e.g. by using inotify).

        Attachments

        Issue Links

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              dtsirogiannis Dimitris Tsirogiannis

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment