Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-21052

Make sure transactions get cleaned if they are aborted before addPartitions is called

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 3.0.0, 3.1.1
    • None
    • Transactions

    Description

      If the transaction is aborted between openTxn and addPartitions and data has been written on the table the transaction manager will think it's an empty transaction and no cleaning will be done.
      This is currently an issue in the streaming API and in micromanaged tables. As proposed by ekoifman this can be solved by:

      • Writing an entry with a special marker to TXN_COMPONENTS at openTxn and when addPartitions is called remove this entry from TXN_COMPONENTS and add the corresponding partition entry to TXN_COMPONENTS.
      • If the cleaner finds and entry with a special marker in TXN_COMPONENTS that specifies that a transaction was opened and it was aborted it must generate jobs for the worker for every possible partition available.

      cc ewohlstadter

      Attachments

        1. HIVE-21052.9.patch
          87 kB
          Jaume M
        2. HIVE-21052.8.patch
          82 kB
          Jaume M
        3. HIVE-21052.7.patch
          731 kB
          Jaume M
        4. HIVE-21052.6.patch
          731 kB
          Jaume M
        5. HIVE-21052.5.patch
          727 kB
          Jaume M
        6. HIVE-21052.4.patch
          727 kB
          Jaume M
        7. HIVE-21052.3.patch
          198 kB
          Jaume M
        8. HIVE-21052.2.patch
          738 kB
          Jaume M
        9. HIVE-21052.12.patch
          94 kB
          Jaume M
        10. HIVE-21052.11.patch
          94 kB
          Jaume M
        11. HIVE-21052.10.patch
          94 kB
          Jaume M
        12. HIVE-21052.1.patch
          738 kB
          Jaume M
        13. Aborted Txn w_Direct Write.pdf
          67 kB
          Eugene Koifman

        Issue Links

          Activity

            People

              jmarhuen Jaume M
              jmarhuen Jaume M
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 12.5h
                  12.5h