[HIVE-21344] CBO: Reduce compilation time in presence of materialized views

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0-alpha-1
• Component/s: Materialized views
• Labels: pull-request-available
• Assignee: Jesús Camacho Rodríguez
• Reporter: Gopal Vijayaraghavan
• Created: 28/Feb/19 07:34
• Updated: 27/Feb/24 22:23
• Resolved: 11/Oct/19 17:15

Description

For every query, getAllValidMaterializedViews still requires a call to metastore to verify that the materializations exist, whether they are outdated or not, etc. Since this is only useful for active-active HS2 deployments, we could take a less aggressive approach and check this information only after rewriting has been triggered. In addition, we could refresh the information in the HS2 registry periodically in a background thread.

// This is not a rebuild, we retrieve all the materializations. In turn, we do not need
// to force the materialization contents to be up-to-date, as this is not a rebuild, and
// we apply the user parameters (HIVE_MATERIALIZED_VIEW_REWRITING_TIME_WINDOW) instead.
materializations = db.getAllValidMaterializedViews(getTablesUsed(basePlan), false, getTxnMgr());

Issue Links

• contains: HIVE-22116 MaterializedView refresh check might return incorrect result when Compaction is run
• is related to: HIVE-22339 Change default time for MVs refresh in registry
• links to: GitHub Pull Request #749