Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-14065

Provide an API for making Hive read-only for a short period

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      HIVE-7973 added a notification log which allows clients to do incremental replication of the Hive metastore. However, it is a challenge to get the initial state of the Hive database. Using existing APIs may give us an inconsistent state. For example, if a Hive table is renamed while we're loading all tables, we may miss that information.

      The easiest way to fix this would be to provide an API for making Hive read-only for a short period. This locking API would come with a timeout so that if the locker failed, the system would not stay down. It would return an ID which uniquely identified the lock instance. The read-only lock itself could be implemented by taking all the ZooKeeper locks. The RPC for removing the lock would return back a status indicating whether the lock had timed out before being removed or not. If it had timed out, we could retry our snapshot loading process with a longer timeout period.

      Attachments

        Issue Links

          Activity

            People

              cmccabe Colin McCabe
              cmccabe Colin McCabe
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated: