Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-21052

Make sure transactions get cleaned if they are aborted before addPartitions is called

Log workAgile BoardRank to TopRank to BottomBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 3.0.0, 3.1.1
    • None
    • Transactions

    Description

      If the transaction is aborted between openTxn and addPartitions and data has been written on the table the transaction manager will think it's an empty transaction and no cleaning will be done.
      This is currently an issue in the streaming API and in micromanaged tables. As proposed by Eugene Koifman this can be solved by:

      • Writing an entry with a special marker to TXN_COMPONENTS at openTxn and when addPartitions is called remove this entry from TXN_COMPONENTS and add the corresponding partition entry to TXN_COMPONENTS.
      • If the cleaner finds and entry with a special marker in TXN_COMPONENTS that specifies that a transaction was opened and it was aborted it must generate jobs for the worker for every possible partition available.

      cc Eric Wohlstadter

      Attachments

        1. Aborted Txn w_Direct Write.pdf
          67 kB
          Eugene Koifman
        2. HIVE-21052.1.patch
          738 kB
          Jaume M
        3. HIVE-21052.10.patch
          94 kB
          Jaume M
        4. HIVE-21052.11.patch
          94 kB
          Jaume M
        5. HIVE-21052.12.patch
          94 kB
          Jaume M
        6. HIVE-21052.2.patch
          738 kB
          Jaume M
        7. HIVE-21052.3.patch
          198 kB
          Jaume M
        8. HIVE-21052.4.patch
          727 kB
          Jaume M
        9. HIVE-21052.5.patch
          727 kB
          Jaume M
        10. HIVE-21052.6.patch
          731 kB
          Jaume M
        11. HIVE-21052.7.patch
          731 kB
          Jaume M
        12. HIVE-21052.8.patch
          82 kB
          Jaume M
        13. HIVE-21052.9.patch
          87 kB
          Jaume M

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            jmarhuen Jaume M Assign to me
            jmarhuen Jaume M
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - Not Specified
              Not Specified
              Remaining:
              Remaining Estimate - 0h
              0h
              Logged:
              Time Spent - 12.5h
              12.5h

              Slack

                Issue deployment