Hive
  1. Hive
  2. HIVE-2467

HA Support for Metastore Server

    Details

      Description

      We require HA deployment for metastore server for HCatalog:

      • Multiple server instances run behind VIP
      • Database provides HA

      Metastore server instances will need to be able to share any state required for VIP outside RDBMS. As of Hive 0.8 affected conversational state that needs to support VIP/HA setup is limited to current delegation tokens. Is this correct?

      We are planning to use ZooKeeper to share current delegation tokens and master keys between nodes of the VIP. ZK is already (optionally) used by Hive for concurrency control. Access to ZK would be limited on the network level or in the future, when ZooKeeper supports security, through Kerberos, similar to NN access.

      Currently Hive taps into Hadoop core security delegation token support through extension of
      org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager<TokenIdent>

      A solution could amend the Hive specific extension to support:

      • Pluggable delegation token and master key store (ZooKeeper as alternative for in-memory AbstractDelegationTokenSecretManager)
      • Delegation token retrieval from token store when not found in memory (wrap/extend retrievePassword(...))
      • Cancellation of token in token store
      • Purging of expired tokens from token store

      http://www.mail-archive.com/hcatalog-user@incubator.apache.org/msg00053.html

      1. HIVE-2467.patch
        49 kB
        Thomas Weise
      2. HIVE-2467.2.patch
        48 kB
        Thomas Weise

        Issue Links

          Activity

          Thomas Weise created issue -
          Thomas Weise made changes -
          Field Original Value New Value
          Attachment HIVE-2467.patch [ 12502218 ]
          Thomas Weise made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Affects Version/s 0.9.0 [ 12317742 ]
          Fix Version/s 0.9.0 [ 12317742 ]
          John Sichi made changes -
          Assignee Thomas Weise [ thw ]
          Thomas Weise made changes -
          Component/s Security [ 12313866 ]
          Component/s Server Infrastructure [ 12312588 ]
          Component/s Metastore [ 12312584 ]
          Thomas Weise made changes -
          Link This issue depends on HIVE-1696 [ HIVE-1696 ]
          Thomas Weise made changes -
          Component/s Metastore [ 12312584 ]
          Thomas Weise made changes -
          Attachment HIVE-2467.2.patch [ 12504000 ]
          Thomas Weise made changes -
          Link This issue relates to HADOOP-7829 [ HADOOP-7829 ]
          Ashutosh Chauhan made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Thomas Weise made changes -
          Link This issue is related to ZOOKEEPER-938 [ ZOOKEEPER-938 ]
          Carl Steinbach made changes -
          Fix Version/s 0.8.0 [ 12316178 ]
          Carl Steinbach made changes -
          Fix Version/s 0.9.0 [ 12317742 ]
          Carl Steinbach made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Thomas Weise made changes -
          Link This issue incorporates HIVE-2712 [ HIVE-2712 ]
          Gavin made changes -
          Link This issue depends on HIVE-1696 [ HIVE-1696 ]
          Gavin made changes -
          Link This issue depends upon HIVE-1696 [ HIVE-1696 ]

            People

            • Assignee:
              Thomas Weise
              Reporter:
              Thomas Weise
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development