Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-106

Make Schema and Table SPIs simpler to implement, and make them re-usable across connections

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:

      Description

      Currently you can only create a Schema when you have a QueryProvider, and the usual implementation of QueryProvider is an OptiqConnection. Thus you have to build a Schema for each connection. The reason for this is that each Table implements Queryable, and therefore needs to contain sufficient information that you can call enumerator() on any Table.

      Also, if a Schema is to be long-lived, it cannot own a type-factory. One of the important functions of RelDataTypeFactory is to convert types to canonical form: for two instances of RelDataType t1 and t2 created by the same type factory, t1.equals(t2) if and only if t1 == t2. But if type factory is long-lived, the list of canonized types can build up and behave like a memory leak.

      Proposed changes:

      • remove Schema.getTypeFactory() and Schema.getQueryProvider() methods;
      • Table would no longer implement Queryable;
      • add a new method Table.toQueryable(QueryProvider);
      • add a interface Type, with method RelDataType Type.toOptiqType(RelDataTypeFactory)
      • change Table.getRowType() to return Type, not RelDataType.

      The changes to the type system allow tables to represent their type in whichever way is best for their domain. The toOptiqType method is called within the context of a statement, and can therefore use a type factory to canonize the RelDataType objects it returns. But optiq does not require or expect that Type instances are canonical.

      ---------------- Imported from GitHub ----------------
      Url: https://github.com/julianhyde/optiq/issues/106
      Created by: julianhyde
      Labels:
      Created at: Tue Dec 17 21:34:07 CET 2013
      State: closed

        Activity

        Hide
        github-import GitHub Import added a comment -

        [Date: Wed Dec 18 16:34:37 CET 2013, Author: jacques-n]

        Couple of thoughts/questions...

        • While should a table even implement a toQueryable method. It seems like a QueryProvider should implement a .toQueryable(Table) method so that tables don't even need to think about whether they can be queried.
        • I don't really understand the type changes. Why does Type need to be convertible to an OptiqType? Maybe an example of exactly the pluggability you envision here would be helpful. I'm thinking this makes sense as Drill has its own internal type representation but I'm not sure you'd always have a 1:1 between the two realms.
        Show
        github-import GitHub Import added a comment - [Date: Wed Dec 18 16:34:37 CET 2013, Author: jacques-n ] Couple of thoughts/questions... While should a table even implement a toQueryable method. It seems like a QueryProvider should implement a .toQueryable(Table) method so that tables don't even need to think about whether they can be queried. I don't really understand the type changes. Why does Type need to be convertible to an OptiqType? Maybe an example of exactly the pluggability you envision here would be helpful. I'm thinking this makes sense as Drill has its own internal type representation but I'm not sure you'd always have a 1:1 between the two realms.
        Hide
        github-import GitHub Import added a comment -

        [Date: Wed Dec 25 22:04:02 CET 2013, Author: julianhyde]

        Re question 1. The `QueryProvider` is provided by Optiq, so it wouldn't necessarily know how to deal with the particular type of `Table`. I now think that the method should be moved to a subclass of `Table`:

        ```java
        public interface QueryableTable extends Table

        { <T> Queryable toQueryable(QueryProvider queryProvider); }

        ```

        Re question 2. I'd like people to be able to build an AST for their particular query language (e.g. DrillQL or SQL) and to be able to validate and type-check that AST within the type system of that language. So doesn't that mean that a table needs two types: one in its native type system (e.g. Drill) and another in Optiq's type system. With my proposed Type interface, you could get one type and then transform it to the other by calling the `toOptiqType` method.

        An alternative would be for `Table` to have a `RelDataType getType(RelDataTypeFactory typeFactory)` method only. A sub-class of Table could have an extra type method, e.g. `DrillTable` could have `DrillType getDrillType()`, but most sub-classes would not go to the trouble.

        Show
        github-import GitHub Import added a comment - [Date: Wed Dec 25 22:04:02 CET 2013, Author: julianhyde ] Re question 1. The `QueryProvider` is provided by Optiq, so it wouldn't necessarily know how to deal with the particular type of `Table`. I now think that the method should be moved to a subclass of `Table`: ```java public interface QueryableTable extends Table { <T> Queryable toQueryable(QueryProvider queryProvider); } ``` Re question 2. I'd like people to be able to build an AST for their particular query language (e.g. DrillQL or SQL) and to be able to validate and type-check that AST within the type system of that language. So doesn't that mean that a table needs two types: one in its native type system (e.g. Drill) and another in Optiq's type system. With my proposed Type interface, you could get one type and then transform it to the other by calling the `toOptiqType` method. An alternative would be for `Table` to have a `RelDataType getType(RelDataTypeFactory typeFactory)` method only. A sub-class of Table could have an extra type method, e.g. `DrillTable` could have `DrillType getDrillType()`, but most sub-classes would not go to the trouble.
        Hide
        github-import GitHub Import added a comment -

        [Date: Fri Jan 03 18:38:34 CET 2014, Author: julianhyde]

        More progress. Table will lose its "String getName()" and "Schema getSchema()" methods. It's up to Optiq to remember structure, not the SPI. It will provided these when needed, e.g. as parameters to the `QueryableTable.getExpression(SchemaPlus schema, String tableName)` method.

        I introduced `SchemaPlus`. It wraps the `Schema` written by the user, and has extra methods, and extra state to allow caching. Its parent and children are also SchemaPlus; i.e. schemas are wrapped up and down the entire hierarchy.

        Also rationalized the methods for getting tables, table-functions and sub-schemas in a schema. They are getTableNames(), getTable(String), getTableFunctionNames(), getTableFunctions(String name), getSubSchemaNames(), getSubSchema(String name).

        Show
        github-import GitHub Import added a comment - [Date: Fri Jan 03 18:38:34 CET 2014, Author: julianhyde ] More progress. Table will lose its "String getName()" and "Schema getSchema()" methods. It's up to Optiq to remember structure, not the SPI. It will provided these when needed, e.g. as parameters to the `QueryableTable.getExpression(SchemaPlus schema, String tableName)` method. I introduced `SchemaPlus`. It wraps the `Schema` written by the user, and has extra methods, and extra state to allow caching. Its parent and children are also SchemaPlus; i.e. schemas are wrapped up and down the entire hierarchy. Also rationalized the methods for getting tables, table-functions and sub-schemas in a schema. They are getTableNames(), getTable(String), getTableFunctionNames(), getTableFunctions(String name), getSubSchemaNames(), getSubSchema(String name).

          People

          • Assignee:
            Unassigned
            Reporter:
            github-import GitHub Import
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development