Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-1104

Materialized view support for Cassandra adapter

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.7.0
    • Component/s: None
    • Labels:
      None

      Description

      I managed to get support for a subset of Cassandra materialized views. ( Thanks owed to some hints from Maryann Xue!)

      The one thing I noticed so far which is not supported is UUID literals in the WHERE clauses in the Cassandra view definition. However, I don't think that's a common use case. The adapter will log a warning if it encounters a view that can't be processed. At the very least, the view will be exposed as a table to Calcite (it wasn't in the previous version of the adapter).

      As per CALCITE-1101, there is probably a better way to hook in and add these views. I thought I would still put this up for now. I'd still like to add some tests before considering this viable.

        Issue Links

          Activity

          Hide
          michaelmior Michael Mior added a comment -

          PR on GitHub for review.

          Show
          michaelmior Michael Mior added a comment - PR on GitHub for review.
          Hide
          julianhyde Julian Hyde added a comment -

          Upgrading Guava is tricky, so let's make that a separate JIRA case and commit. I've created CALCITE-1111.

          Show
          julianhyde Julian Hyde added a comment - Upgrading Guava is tricky, so let's make that a separate JIRA case and commit. I've created CALCITE-1111 .
          Hide
          julianhyde Julian Hyde added a comment -

          Michael Mior, Are we still waiting on tests?

          Show
          julianhyde Julian Hyde added a comment - Michael Mior , Are we still waiting on tests?
          Hide
          michaelmior Michael Mior added a comment -

          Yes. I haven't been pushing on this since it's blocked on the Guava upgrade anyway. I should be able to make some movement later this week.

          Show
          michaelmior Michael Mior added a comment - Yes. I haven't been pushing on this since it's blocked on the Guava upgrade anyway. I should be able to make some movement later this week.
          Hide
          julianhyde Julian Hyde added a comment -

          CALCITE-1111 is fixed now. Can we move forward with this one?

          Show
          julianhyde Julian Hyde added a comment - CALCITE-1111 is fixed now. Can we move forward with this one?
          Hide
          michaelmior Michael Mior added a comment -

          Confirmed that this allows the upgrade to the new version of the Cassandra driver. I'll try to add those tests in tomorrow. Will probably require a PR to add a materialized view to the sample dataset in the VM.

          Show
          michaelmior Michael Mior added a comment - Confirmed that this allows the upgrade to the new version of the Cassandra driver. I'll try to add those tests in tomorrow. Will probably require a PR to add a materialized view to the sample dataset in the VM.
          Hide
          michaelmior Michael Mior added a comment -

          While trying to put a reasonable set of tests together, I encountered the following warning that prevents Calcite from using a materialized view which includes "IS NOT NULL" and a query which tests the same column against a string constant

          [main] WARN - Support for checking =(CAST($2):CHAR(3) CHARACTER SET "ISO-8859-1" COLLATE "ISO-8859-1$en_US$primary", 'fOGctyIDES') => IS NOT NULL($2) is not there

          Any ideas how this can be resolved? It seems like deciding that a non-null string constant is not null should be trivial but I'm sure I'm missing some complexities.

          Show
          michaelmior Michael Mior added a comment - While trying to put a reasonable set of tests together, I encountered the following warning that prevents Calcite from using a materialized view which includes "IS NOT NULL" and a query which tests the same column against a string constant [main] WARN - Support for checking =(CAST($2):CHAR(3) CHARACTER SET "ISO-8859-1" COLLATE "ISO-8859-1$en_US$primary", 'fOGctyIDES') => IS NOT NULL($2) is not there Any ideas how this can be resolved? It seems like deciding that a non-null string constant is not null should be trivial but I'm sure I'm missing some complexities.
          Hide
          julianhyde Julian Hyde added a comment -

          Amogh Margoor, You added the message "support for checking ... => ... is not there" in https://git1-us-west.apache.org/repos/asf?p=calcite.git;a=commit;h=9db3b023 while fixing CALCITE-925. Can you comment?

          Show
          julianhyde Julian Hyde added a comment - Amogh Margoor , You added the message "support for checking ... => ... is not there" in https://git1-us-west.apache.org/repos/asf?p=calcite.git;a=commit;h=9db3b023 while fixing CALCITE-925 . Can you comment?
          Hide
          amargoor Amogh Margoor added a comment -

          Julian Hyde Michael Mior Will take a look. Looks like a valid implication and if it's not supported currently, I will add the support for it in RexImplicationChecker.

          Show
          amargoor Amogh Margoor added a comment - Julian Hyde Michael Mior Will take a look. Looks like a valid implication and if it's not supported currently, I will add the support for it in RexImplicationChecker.
          Hide
          michaelmior Michael Mior added a comment -

          Awesome, thanks! Let me know if there's anything I can do to help. This happens to be particularly relevant here because Cassandra requires that materialized views be defined by a query which cannot produce NULL values for the primary keys.

          Show
          michaelmior Michael Mior added a comment - Awesome, thanks! Let me know if there's anything I can do to help. This happens to be particularly relevant here because Cassandra requires that materialized views be defined by a query which cannot produce NULL values for the primary keys.
          Hide
          julianhyde Julian Hyde added a comment -

          Michael Mior, Amogh Margoor, Rajat: Since these issues are not directly related to Cassandra I think you should create a new JIRA case, possibly including the issues in https://github.com/qubole/quark/issues/27. Michael, you should decide whether the new JIRA case blocks this one (1104).

          Show
          julianhyde Julian Hyde added a comment - Michael Mior , Amogh Margoor , Rajat: Since these issues are not directly related to Cassandra I think you should create a new JIRA case, possibly including the issues in https://github.com/qubole/quark/issues/27 . Michael, you should decide whether the new JIRA case blocks this one (1104).
          Hide
          amargoor Amogh Margoor added a comment -

          Julian Hyde Michael Mior Currently we are supporting only comparison operators (check SqlKind.COMPARISION). Based on request for Cassandra, I have extended the support for IS_NULL and IS_NOT_NULL: JIRA - CALCITE-1130, PR: https://github.com/apache/calcite/pull/207. Michael, Can you check if my patch unblocks you ?

          Will take care of issues raised by Rajat Venkatesh in a separate JIRA and this should not be hopefully blocking this JIRA (1104)

          Show
          amargoor Amogh Margoor added a comment - Julian Hyde Michael Mior Currently we are supporting only comparison operators (check SqlKind.COMPARISION). Based on request for Cassandra, I have extended the support for IS_NULL and IS_NOT_NULL: JIRA - CALCITE-1130 , PR: https://github.com/apache/calcite/pull/207 . Michael, Can you check if my patch unblocks you ? Will take care of issues raised by Rajat Venkatesh in a separate JIRA and this should not be hopefully blocking this JIRA (1104)
          Hide
          michaelmior Michael Mior added a comment -

          Now that 1130 is cleared up, this is almost good to go. I have a PR in for the test VM to upgrade and I still just need to write a couple unit tests.

          Show
          michaelmior Michael Mior added a comment - Now that 1130 is cleared up, this is almost good to go. I have a PR in for the test VM to upgrade and I still just need to write a couple unit tests.
          Hide
          michaelmior Michael Mior added a comment -

          Sorry for the delay. Added tests and updated the PR on GitHub.

          Show
          michaelmior Michael Mior added a comment - Sorry for the delay. Added tests and updated the PR on GitHub.
          Hide
          julianhyde Julian Hyde added a comment -

          +1 looks good, tests pass (on a VM created using https://github.com/vlsi/calcite-test-dataset/pull/6)

          Vladimir Sitnikov, Please let us know when that PR is accepted.

          Show
          julianhyde Julian Hyde added a comment - +1 looks good, tests pass (on a VM created using https://github.com/vlsi/calcite-test-dataset/pull/6 ) Vladimir Sitnikov , Please let us know when that PR is accepted.
          Hide
          michaelmior Michael Mior added a comment -

          PR has been merged.

          Show
          michaelmior Michael Mior added a comment - PR has been merged.
          Hide
          julianhyde Julian Hyde added a comment -
          Show
          julianhyde Julian Hyde added a comment - Fixed in http://git-wip-us.apache.org/repos/asf/calcite/commit/aacb2375 . Thanks for the PR, Michael Mior !
          Hide
          michaelmior Michael Mior added a comment -

          Thanks for the merge!

          Show
          michaelmior Michael Mior added a comment - Thanks for the merge!
          Hide
          julianhyde Julian Hyde added a comment -

          Resolved in release 1.7.0 (2016-03-22).

          Show
          julianhyde Julian Hyde added a comment - Resolved in release 1.7.0 (2016-03-22).

            People

            • Assignee:
              julianhyde Julian Hyde
              Reporter:
              michaelmior Michael Mior
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development