Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-12636

Ensure that all queries (with DbTxnManager) run in a transaction

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 1.0.0
    • 3.0.0
    • Transactions
    • None

    Description

      Assuming Hive is using DbTxnManager
      Currently (as of this writing only auto commit mode is supported), only queries that write to an Acid table start a transaction.
      Read-only queries don't open a txn but still acquire locks.
      This makes internal structures confusing/odd.
      The are constantly 2 code paths to deal with which is inconvenient and error prone.

      Also, a txn id is convenient "handle" for all locks/resources within a txn.
      Doing thing would mean the client no longer needs to track locks that it acquired. This enables further improvements to metastore side of Acid.

      1. add metastore call to openTxn() and acquireLocks() in a single call. this it to make sure perf doesn't degrade for read-only query. (Would also be useful for auto commit write queries)
      2. Should RO queries generate txn ids from the same sequence? (they could for example use negative values of a different sequence). Txnid is part of the delta/base file name. Currently it's 7 digits. If we use the same sequence, we'll exceed 7 digits faster. (possible upgrade issue). On the other hand there is value in being able to pick txn id and commit timestamp out of the same logical sequence.

      Attachments

        1. HIVE-12636.01.patch
          131 kB
          Eugene Koifman
        2. HIVE-12636.02.patch
          162 kB
          Eugene Koifman
        3. HIVE-12636.03.patch
          167 kB
          Eugene Koifman
        4. HIVE-12636.04.patch
          178 kB
          Eugene Koifman
        5. HIVE-12636.05.patch
          179 kB
          Eugene Koifman
        6. HIVE-12636.06.patch
          179 kB
          Eugene Koifman
        7. HIVE-12636.07.patch
          180 kB
          Eugene Koifman
        8. HIVE-12636.09.patch
          189 kB
          Eugene Koifman
        9. HIVE-12636.10.patch
          190 kB
          Eugene Koifman
        10. HIVE-12636.12.patch
          193 kB
          Eugene Koifman
        11. HIVE-12636.13.patch
          205 kB
          Eugene Koifman
        12. HIVE-12636.17.patch
          190 kB
          Eugene Koifman

        Issue Links

          There are no Sub-Tasks for this issue.

          Activity

            People

              ekoifman Eugene Koifman
              ekoifman Eugene Koifman
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: