Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-16867

Extend shared scan optimizer to reuse computation from other operators

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: 3.0.0
    • Component/s: Physical Optimizer
    • Labels:
    • Target Version/s:

      Description

      Follow-up of the work in HIVE-16602.

      HIVE-16602 introduced an optimization that identifies scans on input tables that can be merged so the data is read only once.
      This extension to that rule allows to reuse the computation that is done in the work containing those scans. In particular, we traverse both parts of the plan upstream and reuse the operators if possible.
      Currently, the optimizer will not go beyond the output edge(s) of that work. Follow-up extensions might remove this limitation.

        Attachments

        1. HIVE-16867.patch
          49 kB
          Jesus Camacho Rodriguez
        2. HIVE-16867.04.patch
          664 kB
          Jesus Camacho Rodriguez
        3. HIVE-16867.03.patch
          585 kB
          Jesus Camacho Rodriguez
        4. HIVE-16867.02.patch
          581 kB
          Jesus Camacho Rodriguez
        5. HIVE-16867.01.patch
          574 kB
          Jesus Camacho Rodriguez

          Issue Links

            Activity

              People

              • Assignee:
                jcamachorodriguez Jesus Camacho Rodriguez
                Reporter:
                jcamachorodriguez Jesus Camacho Rodriguez
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: