Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-584

Clean up global and ThreadLocal variables in Hive

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.3.0
    • 0.5.0
    • None
    • None
    • Reviewed
    • HIVE-584. Clean up global and ThreadLocal variables in Hive. (Yonqqiang He via zshao)

    Description

      Currently in Hive code there are several global and ThreadLocal variables that need to be cleaned.

      Specifically, the following classes are involved:
      1. HiveConf: contains hive configurations (and a classloader)
      2. Hive class: contains a static member Hive db. Hive class contains a member HiveConf conf, as well as a ThreadLocal storage of IMetaStoreClient.
      3. SessionState: contains a static ThreadLocal storage of SessionState. SessionState class contains a Hive db, a HiveConf conf, a history logger, and a bunch of standard input/output streams
      4. CliSessionState: SessionState plus some command options and the command file name.
      5. All classes that try to get Hive db or HiveConf from global static Hive db, or SessionState.

      There are several problems with the current design. To name a few:
      1. SessionState instances are ThreadLocal, but SessionState contains Hive db which also contains ThreadLocal storage. Not sure a db can be shared across different threads or not? What is the global static Hive db?
      2. We pass HiveConf and Hive db in two ways to classes like Task: Sometimes through initialize(), sometimes through SessionState. This complicates the code a lot. It's hard to know which HiveConf and which db we should use.

      We need to think about a better way to do it.

      Attachments

        1. hive-584-2009-9-10.patch
          6 kB
          He Yongqiang
        2. hive-584-2009-9-11.patch
          6 kB
          He Yongqiang

        Issue Links

          blocks

          New Feature - A new feature of the product, which has yet to be developed. HIVE-805 Session level metastore

          • Major - Major loss of function.
          • Open

          New Feature - A new feature of the product, which has yet to be developed. HIVE-675 Add Database/Schema support to Hive QL

          • Major - Major loss of function.
          • Closed

          Test - A new unit, integration or system test. HIVE-80 Add testcases for concurrent query execution

          • Critical - Crashes, loss of data, severe memory leak.
          • Open
          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. HIVE-574 Hive should use ClassLoader from hadoop Configuration

          • Major - Major loss of function.
          • Closed
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-210 Refactor HiveConf variables into two categories (query and session level)

          • Major - Major loss of function.
          • Open

          Activity

            People

              Unassigned Unassigned
              zshao Zheng Shao
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: