Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-558

Add BINDABLE convention and make it the default convention for query root

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      The default convention is currently ENUMERABLE, which means that to execute a query, and in particular to use the JDBC driver, RelNodes need to be able to convert themselves to code. This introduces a lot of dependencies that make CALCITE-466 difficult to solve.

      We propose to add a new convention that can be more easily implemented by a RelNode, using an interpreter or otherwise.

        Issue Links

          Activity

          Hide
          julianhyde Julian Hyde added a comment - - edited

          Progress: Commit http://git-wip-us.apache.org/repos/asf/calcite/commit/66cfb120 adds bindable convention but does not make it default.

          Show
          julianhyde Julian Hyde added a comment - - edited Progress: Commit http://git-wip-us.apache.org/repos/asf/calcite/commit/66cfb120 adds bindable convention but does not make it default.
          Hide
          vladimirsitnikov Vladimir Sitnikov added a comment -

          Julian Hyde, I am looking into all that stuff and I've a question: does linq4j.AbstractNode.evaluate count as an interpreter?

          Show
          vladimirsitnikov Vladimir Sitnikov added a comment - Julian Hyde , I am looking into all that stuff and I've a question: does linq4j.AbstractNode.evaluate count as an interpreter?
          Hide
          julianhyde Julian Hyde added a comment -

          I can see where you're coming from. Technically yes, it's an interpreter. But it doesn't help solve the problem that you have to translate relational algebra into a very different language (iterators and lots of lambdas) to get your query to work.

          We've had lots of people say 'yuck' when they find that they have to generate java code in order to write a connector to a data source like mongo. I want to make it easier for them.

          By the way, another nice thing about a true relational interpreter is that you could envisage building a debugger on top of it, so you could - say - single-step the rows flowing through each operator.

          Show
          julianhyde Julian Hyde added a comment - I can see where you're coming from. Technically yes, it's an interpreter. But it doesn't help solve the problem that you have to translate relational algebra into a very different language (iterators and lots of lambdas) to get your query to work. We've had lots of people say 'yuck' when they find that they have to generate java code in order to write a connector to a data source like mongo. I want to make it easier for them. By the way, another nice thing about a true relational interpreter is that you could envisage building a debugger on top of it, so you could - say - single-step the rows flowing through each operator.
          Hide
          vladimirsitnikov Vladimir Sitnikov added a comment - - edited

          We've had lots of people say 'yuck' when they find that they have to generate java code in order to write a connector to a data source like mongo.

          Current MongoToEnumerableConverter is like 100 lines long.
          It does not look like a "yuck".

          Well, it does, however the major part of the mongo code is "passing non literal ArrayList to linq4j".
          "generate java code" part is not more than "tell Calcite to execute MongoTable.aggregate(...)".
          We never suggested moving all the code to linq4j part. On contrary, the idea was to use as little linq4j as it is possible and to upcall to the java.

          The "problem" can easily be solved if something like EnumerableRelImplementor.result(Bindable impl, PhysType) is added.
          If non-numeric, non-string literals were allowed in linq4j, then mongo code would be just "Expressions.call(MONGO_QUERYABLE_AGGREGATE)}}" kind of thing with no other "code generation" stuff.
          It still makes sense to allow "passing of non-literal constant values to linq4j" to simplify the life of those who do generate code.

          So the mongo-to-enumerable adapter could go without any notion of Expression/Expressions/BlockBuilder/etc.
          Do you still think it worths reimplmenting the wheel just for that?

          By the way, another nice thing about a true relational interpreter is that you could envisage building a debugger on top of it,

          The nice part of generation of java code is you can have the generated code in the bug reports.
          Lots of people know how to read java, so it simplifies the analysis.

          Having step-by-step debugging is a nice feature, however I belive IDE's can be tricked to step-by-step through the generated code.

          By the way: when I was implementing window functions support, I copied and pasted the generated code and debugged it in the IDEA from times to times.

          Show
          vladimirsitnikov Vladimir Sitnikov added a comment - - edited We've had lots of people say 'yuck' when they find that they have to generate java code in order to write a connector to a data source like mongo. Current MongoToEnumerableConverter is like 100 lines long. It does not look like a "yuck". Well, it does, however the major part of the mongo code is "passing non literal ArrayList to linq4j". "generate java code" part is not more than "tell Calcite to execute MongoTable.aggregate(...) ". We never suggested moving all the code to linq4j part. On contrary, the idea was to use as little linq4j as it is possible and to upcall to the java. The "problem" can easily be solved if something like EnumerableRelImplementor.result(Bindable impl, PhysType) is added. If non-numeric, non-string literals were allowed in linq4j, then mongo code would be just "Expressions.call(MONGO_QUERYABLE_AGGREGATE)}}" kind of thing with no other "code generation" stuff. It still makes sense to allow "passing of non-literal constant values to linq4j" to simplify the life of those who do generate code. So the mongo-to-enumerable adapter could go without any notion of Expression/Expressions/BlockBuilder/etc . Do you still think it worths reimplmenting the wheel just for that? By the way, another nice thing about a true relational interpreter is that you could envisage building a debugger on top of it, The nice part of generation of java code is you can have the generated code in the bug reports. Lots of people know how to read java, so it simplifies the analysis. Having step-by-step debugging is a nice feature, however I belive IDE's can be tricked to step-by-step through the generated code. By the way: when I was implementing window functions support, I copied and pasted the generated code and debugged it in the IDEA from times to times.
          Hide
          vladimirsitnikov Vladimir Sitnikov added a comment -

          But it doesn't help solve the problem that you have to translate relational algebra into a very different language

          Technically, it does. This translation is already implemented, so AbstractNode.evaluate can bootstrap the interpreter "for free".

          Show
          vladimirsitnikov Vladimir Sitnikov added a comment - But it doesn't help solve the problem that you have to translate relational algebra into a very different language Technically, it does. This translation is already implemented, so AbstractNode.evaluate can bootstrap the interpreter "for free".

            People

            • Assignee:
              julianhyde Julian Hyde
              Reporter:
              julianhyde Julian Hyde
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:

                Development