Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-1566

Better documentation on the use of materialized views

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.13.0
    • Component/s: None
    • Labels:

      Description

      As discussed in CALCITE-1389, it would be helpful to have additional documentation on how Calcite makes use of materialized views.

        Issue Links

          Activity

          Hide
          michaelmior Michael Mior added a comment -

          Julian Hyde Maryann Xue Could you take a look at the first pass at this documentation? I'd appreciate any feedback.

          Show
          michaelmior Michael Mior added a comment - Julian Hyde Maryann Xue Could you take a look at the first pass at this documentation? I'd appreciate any feedback.
          Hide
          maryannxue Maryann Xue added a comment -

          Thank you Michael Mior, for putting together a initial version of materialized view documentation! I'm not sure about the ultimate structure of the document yet, so just a couple of thoughts based on what we have here:
          1. I like the first section talking about materialization usage in adapters, so maybe it would make more sense if we include usage scenarios from other adapters. Think I can do the Phoenix part.
          2. Before jumping into the limitations and technical details of a specific materialized view rewriting rule, it would be nicer to have an introduction of the basic materialized view rewriting idea and some general rewriting rules. List a few query examples with what they would become after rule application.
          3. We could put some important or interesting rewriting rules in sub-sections and talk about some details and limitations, as well as possible future improvement.
          4. I believe right now we have two types of materialized view rewriting rules applied in different stages of query planning, one based on SubstitionVisitor which happens right before volcano planning, the other being RelOptRule which happens during the volcano planning process. So it would be great to talk about their differences and their own preferred scenarios.

          Show
          maryannxue Maryann Xue added a comment - Thank you Michael Mior , for putting together a initial version of materialized view documentation! I'm not sure about the ultimate structure of the document yet, so just a couple of thoughts based on what we have here: 1. I like the first section talking about materialization usage in adapters, so maybe it would make more sense if we include usage scenarios from other adapters. Think I can do the Phoenix part. 2. Before jumping into the limitations and technical details of a specific materialized view rewriting rule, it would be nicer to have an introduction of the basic materialized view rewriting idea and some general rewriting rules. List a few query examples with what they would become after rule application. 3. We could put some important or interesting rewriting rules in sub-sections and talk about some details and limitations, as well as possible future improvement. 4. I believe right now we have two types of materialized view rewriting rules applied in different stages of query planning, one based on SubstitionVisitor which happens right before volcano planning, the other being RelOptRule which happens during the volcano planning process. So it would be great to talk about their differences and their own preferred scenarios.
          Hide
          michaelmior Michael Mior added a comment -

          Thanks for the suggestions!

          Agreed that it would be great to have more scenarios from other adapters. If you could write something for Phoenix that would be appreciated.

          I added a bit more on MaterializedViewSubstitutionVisitor as well as a simple example I pulled from the Javadoc.

          Show
          michaelmior Michael Mior added a comment - Thanks for the suggestions! Agreed that it would be great to have more scenarios from other adapters. If you could write something for Phoenix that would be appreciated. I added a bit more on MaterializedViewSubstitutionVisitor as well as a simple example I pulled from the Javadoc.
          Hide
          julianhyde Julian Hyde added a comment -

          Michael Mior, I have reviewed this and it looks good. Can you please commit it? I have some ideas for extra content (e.g. motivating "what are materialized views and why?", and describing the 3 key problems: designing materialized views, populating them (including incrementally) and rewriting queries to use them), and it sounds as if Maryann Xue does too. We can add our content after you commit.

          Show
          julianhyde Julian Hyde added a comment - Michael Mior , I have reviewed this and it looks good. Can you please commit it? I have some ideas for extra content (e.g. motivating "what are materialized views and why?", and describing the 3 key problems: designing materialized views, populating them (including incrementally) and rewriting queries to use them), and it sounds as if Maryann Xue does too. We can add our content after you commit.
          Hide
          michaelmior Michael Mior added a comment -

          Pushed as 9ef9430c3.

          Show
          michaelmior Michael Mior added a comment - Pushed as 9ef9430c3 .
          Hide
          julianhyde Julian Hyde added a comment -

          Great! I will include it in the web site push for 1.12 later today.

          Show
          julianhyde Julian Hyde added a comment - Great! I will include it in the web site push for 1.12 later today.
          Hide
          julianhyde Julian Hyde added a comment -

          Maryann Xue and Jesus Camacho Rodriguez, As "power users" of materialized views, can you both please make a pass over that document.

          Show
          julianhyde Julian Hyde added a comment - Maryann Xue and Jesus Camacho Rodriguez , As "power users" of materialized views, can you both please make a pass over that document.
          Hide
          jcamachorodriguez Jesus Camacho Rodriguez added a comment -

          Resolved in release 1.13.0 (2017-06-26).

          Show
          jcamachorodriguez Jesus Camacho Rodriguez added a comment - Resolved in release 1.13.0 (2017-06-26).

            People

            • Assignee:
              michaelmior Michael Mior
              Reporter:
              michaelmior Michael Mior
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development