Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-2467

HA Support for Metastore Server

    XMLWordPrintableJSON

Details

    Description

      We require HA deployment for metastore server for HCatalog:

      • Multiple server instances run behind VIP
      • Database provides HA

      Metastore server instances will need to be able to share any state required for VIP outside RDBMS. As of Hive 0.8 affected conversational state that needs to support VIP/HA setup is limited to current delegation tokens. Is this correct?

      We are planning to use ZooKeeper to share current delegation tokens and master keys between nodes of the VIP. ZK is already (optionally) used by Hive for concurrency control. Access to ZK would be limited on the network level or in the future, when ZooKeeper supports security, through Kerberos, similar to NN access.

      Currently Hive taps into Hadoop core security delegation token support through extension of
      org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager<TokenIdent>

      A solution could amend the Hive specific extension to support:

      • Pluggable delegation token and master key store (ZooKeeper as alternative for in-memory AbstractDelegationTokenSecretManager)
      • Delegation token retrieval from token store when not found in memory (wrap/extend retrievePassword(...))
      • Cancellation of token in token store
      • Purging of expired tokens from token store

      http://www.mail-archive.com/hcatalog-user@incubator.apache.org/msg00053.html

      Attachments

        1. HIVE-2467.patch
          49 kB
          Thomas Weise
        2. HIVE-2467.2.patch
          48 kB
          Thomas Weise

        Issue Links

          depends upon

          Sub-task - The sub-task of the issue HIVE-1696 Add delegation token support to metastore

          • Major - Major loss of function.
          • Closed
          incorporates

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-2712 Make ZooKeeper token store ACL configurable

          • Major - Major loss of function.
          • Closed
          is related to

          New Feature - A new feature of the product, which has yet to be developed. ZOOKEEPER-938 Support Kerberos authentication of clients.

          • Major - Major loss of function.
          • Closed
          relates to

          New Feature - A new feature of the product, which has yet to be developed. HIVE-3255 Add DBTokenStore to store Delegation Tokens in DB

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-7829 Delegation token manager should support token store abstraction

          • Major - Major loss of function.
          • Open

          Activity

            People

              thw Thomas Weise
              thw Thomas Weise
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: