Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-17089

make acid 2.0 the default

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 3.0.0
    • Transactions
    • None

    Description

      acid 2.0 is introduced in HIVE-14035. It replaces Update events with a combination of Delete + Insert events. This now makes U=D+I the default (and only) supported acid table type in Hive 3.0.

      The expectation for upgrade is that Major compaction has to be run on all acid tables in the existing Hive cluster and that no new writes to these table take place since the start of compaction (Need to add a mechanism to put a table in read-only mode - this way it can still be read while it's being compacted). Then upgrade to Hive 3.0 can take place.

      Attachments

        1. HIVE-17089.01.patch
          1 kB
          Eugene Koifman
        2. HIVE-17089.03.patch
          81 kB
          Eugene Koifman
        3. HIVE-17089.05.patch
          111 kB
          Eugene Koifman
        4. HIVE-17089.06.patch
          128 kB
          Eugene Koifman
        5. HIVE-17089.07.patch
          174 kB
          Eugene Koifman
        6. HIVE-17089.10.patch
          189 kB
          Eugene Koifman
        7. HIVE-17089.10.patch
          186 kB
          Eugene Koifman
        8. HIVE-17089.11.patch
          203 kB
          Eugene Koifman
        9. HIVE-17089.12.patch
          203 kB
          Eugene Koifman
        10. HIVE-17089.13.patch
          203 kB
          Eugene Koifman
        11. HIVE-17089.14.patch
          204 kB
          Eugene Koifman
        12. HIVE-17089.15.patch
          204 kB
          Eugene Koifman
        13. HIVE-17089.16.patch
          204 kB
          Eugene Koifman

        Issue Links

          blocks

          New Feature - A new feature of the product, which has yet to be developed. HIVE-17204 support un-bucketed tables in acid

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-17320 OrcRawRecordMerger.discoverKeyBounds logic can be simplified

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-17328 Remove special handling for Acid tables wherever possible

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-17282 OrcRawRecordMerger collapseEvents mode should not produce delete events

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-17284 remove OrcRecordUpdater.deleteEventIndexBuilder

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is blocked by

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-14514 OrcRecordUpdater should clone writerOptions when creating delete event writers

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HIVE-16669 Fine tune Compaction to take advantage of Acid 2.0

          • Critical - Crashes, loss of data, severe memory leak.
          • Open

          New Feature - A new feature of the product, which has yet to be developed. SPARK-15348 Hive ACID

          • Major - Major loss of function.
          • Resolved
          is required by

          Bug - A problem which impairs or prevents the functions of the product. HIVE-14516 OrcInputFormat.SplitGenerator.callInternal() can be optimized

          • Major - Major loss of function.
          • Closed
          relates to

          Sub-task - The sub-task of the issue HIVE-16445 enable Acid by default in the parent patch and run build bot

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HIVE-18110 OrcInputFormat.combineOrCreateETLStrategy() doesn't combine if there are deltas

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. ORC-228 Make MemoryManagerImpl.ROWS_BETWEEN_CHECKS configurable

          • Major - Major loss of function.
          • Closed
          requires

          New Feature - A new feature of the product, which has yet to be developed. HIVE-14035 Enable predicate pushdown to delta files created by ACID Transactions

          • Major - Major loss of function.
          • Resolved
          supercedes

          New Feature - A new feature of the product, which has yet to be developed. HIVE-13479 Relax sorting requirement in ACID tables

          • Major - Major loss of function.
          • Open
          links to

          Web Link Review Board

          Activity

            People

              ekoifman Eugene Koifman
              ekoifman Eugene Koifman
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: