Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-14035

Enable predicate pushdown to delta files created by ACID Transactions

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.3.0
    • Component/s: Transactions
    • Labels:
    • Target Version/s:

      Description

      In current Hive version, delta files created by ACID transactions do not allow predicate pushdown if they contain any update/delete events. This is done to preserve correctness when following a multi-version approach during event collapsing, where an update event overwrites an existing insert event.
      This JIRA proposes to split an update event into a combination of a delete event followed by a new insert event, that can enable predicate push down to all delta files without breaking correctness. To support backward compatibility for this feature, this JIRA also proposes to add some sort of versioning to ACID that can allow different versions of ACID transactions to co-exist together.

        Attachments

        1. Design.Document.Improving ACID performance in Hive.01.docx
          24 kB
          Saket Saurabh
        2. Design.Document.Improving ACID performance in Hive.02.docx
          24 kB
          Saket Saurabh
        3. HIVE-14035.02.patch
          94 kB
          Saket Saurabh
        4. HIVE-14035.03.patch
          119 kB
          Saket Saurabh
        5. HIVE-14035.04.patch
          90 kB
          Saket Saurabh
        6. HIVE-14035.05.patch
          94 kB
          Saket Saurabh
        7. HIVE-14035.06.patch
          105 kB
          Saket Saurabh
        8. HIVE-14035.07.patch
          87 kB
          Saket Saurabh
        9. HIVE-14035.08.patch
          117 kB
          Saket Saurabh
        10. HIVE-14035.09.patch
          116 kB
          Saket Saurabh
        11. HIVE-14035.10.patch
          115 kB
          Saket Saurabh
        12. HIVE-14035.11.patch
          152 kB
          Saket Saurabh
        13. HIVE-14035.12.patch
          158 kB
          Saket Saurabh
        14. HIVE-14035.13.patch
          166 kB
          Saket Saurabh
        15. HIVE-14035.14.patch
          197 kB
          Saket Saurabh
        16. HIVE-14035.15.patch
          197 kB
          Saket Saurabh
        17. HIVE-14035.16.patch
          197 kB
          Saket Saurabh
        18. HIVE-14035.17.patch
          197 kB
          Saket Saurabh
        19. HIVE-14035.patch
          91 kB
          Saket Saurabh

          Issue Links

            Activity

              People

              • Assignee:
                saketj Saket Saurabh
                Reporter:
                saketj Saket Saurabh
              • Votes:
                0 Vote for this issue
                Watchers:
                13 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: