Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-11388

Allow ACID Compactor components to run in multiple metastores

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.3.0, 2.1.0
    • Component/s: Transactions
    • Labels:
      None

      Description

      (this description is no loner accurate; see further comments)

      org.apache.hadoop.hive.ql.txn.compactor.Initiator is a thread that runs inside the metastore service to manage compactions of ACID tables. There should be exactly 1 instance of this thread (even with multiple Thrift services).

      This is documented in https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-Configuration but not enforced.

      Should add enforcement, since more than 1 Initiator could cause concurrent attempts to compact the same table/partition - which will not work.

        Attachments

        1. HIVE-11388.2.patch
          26 kB
          Eugene Koifman
        2. HIVE-11388.4.patch
          37 kB
          Eugene Koifman
        3. HIVE-11388.5.patch
          36 kB
          Eugene Koifman
        4. HIVE-11388.6.patch
          38 kB
          Eugene Koifman
        5. HIVE-11388.7.patch
          39 kB
          Eugene Koifman
        6. HIVE-11388.branch-1.patch
          40 kB
          Eugene Koifman
        7. HIVE-11388.patch
          26 kB
          Eugene Koifman

          Issue Links

            Activity

              People

              • Assignee:
                ekoifman Eugene Koifman
                Reporter:
                ekoifman Eugene Koifman
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: