[IMPALA-3446] Materialized views

• Type: Epic
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: Impala 2.5.0
• Component/s: Frontend
• Epic Name: Materialized View
• Assignee: pengdou1990
• Reporter: Marcell Szabo
• Created: 28/Apr/16 13:04
• Updated: 30/Dec/22 04:13

Description

This JIRA is a placeholder for this big topic.
Some user stories I can imagine under the epic:

1. materialized view is a CTAS where the SELECT is saved in the HMS and can be rerun by a simple command
2. materialized view detects that the source data has changed and falls back to be a view instead of SELECT *
3. materialized view detects that the source data has changed and reruns the CTAS automatically
4. ... reruns only the necessary changes (e.g. if only some of the partitions change)
5. Impala and Hive to have common semantics of materialized views (https://issues.apache.org/jira/browse/HIVE-10459)
6. materialized view stores extra statistics that help the optimizer, storing the full resultset is optional
7. query optimizer checks for every query whether part of the query plan can be covered by an existing materialized view

Issue Links

• is related to: HIVE-10459 Add materialized views to Hive
• is related to: IMPALA-10723 Allow basic querying and computing stats on a materialized view