[SPARK-26764] [SPIP] Spark Relational Cache - ASF JIRA

Rank to Top

Rank to Bottom

Attach files

Attach Screenshot

Bulk Copy Attachments

Bulk Move Attachments

Add vote

Voters

Watch issue

Watchers

Create sub-task

Convert to sub-task

Link

Clone

Labels

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: New Feature
Status: Open
Priority: Major
Resolution: Unresolved
Affects Version/s: 3.1.0
Fix Version/s: None
Component/s: SQL
Labels:
None

Description

In modern database systems, relational cache is a common technology to boost ad-hoc queries. While Spark provides cache natively, Spark SQL should be able to utilize the relationship between relations to boost all possible queries. In this SPIP, we will make Spark be able to utilize all defined cached relations if possible, without explicit substitution in user query, as well as keep some user defined cache available in different sessions. Materialized views in many database systems provide similar function.