Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-21344

CBO: Reduce compilation time in presence of materialized views

    XMLWordPrintableJSON

    Details

      Description

      For every query, getAllValidMaterializedViews still requires a call to metastore to verify that the materializations exist, whether they are outdated or not, etc. Since this is only useful for active-active HS2 deployments, we could take a less aggressive approach and check this information only after rewriting has been triggered. In addition, we could refresh the information in the HS2 registry periodically in a background thread.

      // This is not a rebuild, we retrieve all the materializations. In turn, we do not need
      // to force the materialization contents to be up-to-date, as this is not a rebuild, and
      // we apply the user parameters (HIVE_MATERIALIZED_VIEW_REWRITING_TIME_WINDOW) instead.
      materializations = db.getAllValidMaterializedViews(getTablesUsed(basePlan), false, getTxnMgr());
      

        Attachments

        1. mv-get-from-remote.png
          83 kB
          Gopal Vijayaraghavan
        2. calcite-planner-after-fix.svg.zip
          82 kB
          Gopal Vijayaraghavan
        3. HIVE-21344.patch
          48 kB
          Jesus Camacho Rodriguez
        4. HIVE-21344.01.patch
          48 kB
          Jesus Camacho Rodriguez
        5. HIVE-21344.02.patch
          50 kB
          Jesus Camacho Rodriguez
        6. HIVE-21344.03.patch
          56 kB
          Jesus Camacho Rodriguez
        7. HIVE-21344.04.patch
          58 kB
          Jesus Camacho Rodriguez
        8. HIVE-21344.05.patch
          65 kB
          Jesus Camacho Rodriguez

          Issue Links

            Activity

              People

              • Assignee:
                jcamachorodriguez Jesus Camacho Rodriguez
                Reporter:
                gopalv Gopal Vijayaraghavan
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h
                  1h