Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.4.0-incubating
    • Component/s: None
    • Labels:
      None

      Description

      The design for materializations (materialized views) allows for them to be populated on demand (e.g. as an ArrayTable) the first time that the materialization is needed or to already exist, having been populated by the user.

      According to Maryann Xue the latter case does not work. Calcite tries to create a new table with the same name as the current table, then makes the name unique by adding a digit, then goes ahead and populates it.

      We need to (a) allow such materializations to be defined in the JSON model, (b) document how to define them, (c) make them work, (d) add tests.

      Cc James Taylor, as this is related to optimization for Phoenix secondary indexes.

        Activity

        Hide
        maryannxue Maryann Xue added a comment -
            final CalciteConnection connection =
                CalciteMetaImpl.connect(schema.root(), null);
            CalciteSchema.TableEntry tableEntry = schema.getTableBySql(viewSql);
            RelDataType rowType = null;
            if (tableEntry == null) {
              Table table = tableFactory.createTable(schema, viewSql, viewSchemaPath);
              final String tableName = Schemas.uniqueTableName(schema,
                  Util.first(suggestedTableName, "m"));
              tableEntry = schema.add(tableName, table, ImmutableList.of(viewSql));
              Hook.CREATE_MATERIALIZATION.run(tableName);
              rowType = table.getRowType(connection.getTypeFactory());
            }
        

        In the above code from MaterializationService.defineMaterialization(), looks like it tries to get a TableEntry by calling schema.getTableBySql() and if this returns now it will start to create one.
        After I made the change below (just a temporary fix I guess), things worked out fine for me:

        @@ -116,7 +116,10 @@ public MaterializationKey defineMaterialization(final CalciteSchema schema,
         
             final CalciteConnection connection =
                 CalciteMetaImpl.connect(schema.root(), null);
        -    CalciteSchema.TableEntry tableEntry = schema.getTableBySql(viewSql);
        +    CalciteSchema.TableEntry tableEntry = schema.getTable(suggestedTableName, true);
        +    if (tableEntry == null) {
        +      tableEntry = schema.getTableBySql(viewSql);
        +    }
             RelDataType rowType = null;
             if (tableEntry == null) {
               Table table = tableFactory.createTable(schema, viewSql, viewSchemaPath);
        

        I also looked to see if there's any place for customization to add "viewSql --> table" mapping, but didn't seem to find one.

        Show
        maryannxue Maryann Xue added a comment - final CalciteConnection connection = CalciteMetaImpl.connect(schema.root(), null ); CalciteSchema.TableEntry tableEntry = schema.getTableBySql(viewSql); RelDataType rowType = null ; if (tableEntry == null ) { Table table = tableFactory.createTable(schema, viewSql, viewSchemaPath); final String tableName = Schemas.uniqueTableName(schema, Util.first(suggestedTableName, "m" )); tableEntry = schema.add(tableName, table, ImmutableList.of(viewSql)); Hook.CREATE_MATERIALIZATION.run(tableName); rowType = table.getRowType(connection.getTypeFactory()); } In the above code from MaterializationService.defineMaterialization(), looks like it tries to get a TableEntry by calling schema.getTableBySql() and if this returns now it will start to create one. After I made the change below (just a temporary fix I guess), things worked out fine for me: @@ -116,7 +116,10 @@ public MaterializationKey defineMaterialization( final CalciteSchema schema, final CalciteConnection connection = CalciteMetaImpl.connect(schema.root(), null ); - CalciteSchema.TableEntry tableEntry = schema.getTableBySql(viewSql); + CalciteSchema.TableEntry tableEntry = schema.getTable(suggestedTableName, true ); + if (tableEntry == null ) { + tableEntry = schema.getTableBySql(viewSql); + } RelDataType rowType = null ; if (tableEntry == null ) { Table table = tableFactory.createTable(schema, viewSql, viewSchemaPath); I also looked to see if there's any place for customization to add "viewSql --> table" mapping, but didn't seem to find one.
        Show
        julianhyde Julian Hyde added a comment - Work in progress at https://github.com/julianhyde/incubator-calcite/tree/761-pre-pop .
        Hide
        maryannxue Maryann Xue added a comment -

        This is patch based on branch 761-pre-pop. Could not go with the original test with foodmart schema coz the pre-populated materialization was defined in different schema other than foodmart and the table could not be identified as existing. Changed to HR schema which is updatable and can have a materialized view defined in itself.

        Show
        maryannxue Maryann Xue added a comment - This is patch based on branch 761-pre-pop. Could not go with the original test with foodmart schema coz the pre-populated materialization was defined in different schema other than foodmart and the table could not be identified as existing. Changed to HR schema which is updatable and can have a materialized view defined in itself.
        Hide
        julianhyde Julian Hyde added a comment -

        Thanks for the patch. I have added your patch to https://github.com/julianhyde/incubator-calcite/tree/761-pre-pop and made some fixes. Testing now.

        Show
        julianhyde Julian Hyde added a comment - Thanks for the patch. I have added your patch to https://github.com/julianhyde/incubator-calcite/tree/761-pre-pop and made some fixes. Testing now.
        Hide
        julianhyde Julian Hyde added a comment -
        Show
        julianhyde Julian Hyde added a comment - Fixed in http://git-wip-us.apache.org/repos/asf/incubator-calcite/commit/4b60b9bf . Thanks for the patch, Maryann Xue !
        Hide
        jnadeau Jacques Nadeau added a comment -

        Resolved in release 1.4.0-incubating (2015-08-23)

        Show
        jnadeau Jacques Nadeau added a comment - Resolved in release 1.4.0-incubating (2015-08-23)

          People

          • Assignee:
            julianhyde Julian Hyde
            Reporter:
            julianhyde Julian Hyde
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development