Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-26764

[SPIP] Spark Relational Cache

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.1.0
    • Fix Version/s: None
    • Component/s: SQL
    • Labels:
      None

      Description

      In modern database systems, relational cache is a common technology to boost ad-hoc queries. While Spark provides cache natively, Spark SQL should be able to utilize the relationship between relations to boost all possible queries. In this SPIP, we will make Spark be able to utilize all defined cached relations if possible, without explicit substitution in user query, as well as keep some user defined cache available in different sessions. Materialized views in many database systems provide similar function.

        Attachments

        1. Relational+Cache+SPIP.pdf
          413 kB
          Adrian Wang

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                adrian-wang Adrian Wang
              • Votes:
                2 Vote for this issue
                Watchers:
                19 Start watching this issue

                Dates

                • Created:
                  Updated: