[HIVE-24274] Implement Query Text based MaterializedView rewrite

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Labels: pull-request-available
• Assignee: Krisztian Kasa
• Reporter: Krisztian Kasa
• Created: 14/Oct/20 14:34
• Updated: 11/Dec/23 15:55
• Resolved: 15/Dec/20 05:37

Description

Besides the way queries are currently rewritten to use materialized views in Hive this project provides an alternative:
Compare the query text with the materialized views query text stored. If we found a match the original query's logical plan can be replaced by a scan on the materialized view.

• Only materialized views which are enabled to rewrite can participate
• Use existing HiveMaterializedViewsRegistry through Hive object by adding a lookup method by query text.
• There might be more than one materialized views which have the same query text. In this case chose the first valid one.
• Validation can be done by calling Hive.validateMaterializedViewsFromRegistry()
• The scope of this first patch is rewriting queries which entire text can be matched only.
• Use the expanded query text (fully qualified column and table names) for comparing

Issue Links

• causes: HIVE-27948 Wrong results when using materialized views with non-deterministic/dynamic functions
• contains: HIVE-24027 Add support for `intersect` keyword in MV
• is related to: HIVE-25240 Query Text based MaterializedView rewrite of subqueries
• relates to: HIVE-24547 Fix acid_vectorization_original
• relates to: HIVE-24820 MaterializedViewCache enables adding multiple entries of the same Materialization instance
• relates to: HIVE-24822 Materialized View rebuild loses materializationTime property value
• relates to: HIVE-24434 Filter out materialized views for rewriting if plan pattern is not allowed
• links to: GitHub Pull Request #1561
• links to: GitHub Pull Request #1706