Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-821

Frameworks gives NPE when FrameworkConfig has no default schema

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.4.0-incubating
    • Component/s: core
    • Labels:
      None

      Description

      During testing of 1.4-SNAPSHOT on Hive, we hit the Exception with the following stacktrace in all our tests:

      java.lang.NullPointerException
              at org.apache.calcite.jdbc.CalciteSchema.from(CalciteSchema.java:316)
              at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:109)
              at org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:880)
              at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:147)
              at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:105)
              at org.apache.hadoop.hive.ql.parse.CalcitePlanner.getOptimizedAST(CalcitePlanner.java:614)
              at org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:249)
              at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10130)
              at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:209)
              at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:240)
      ...
      

      The problem is that the FrameworkConfig defaultSchema is null. Thus, a NPE is thrown after hitting this piece of code in Frameworks:

      final CalciteSchema schema =
        CalciteSchema.from(config.getDefaultSchema());
      

      The apply method was changed in CALCITE-748.

        Activity

        Hide
        jcamachorodriguez Jesus Camacho Rodriguez added a comment - - edited

        Julian Hyde, what do you think? Why did that method change, instead of using rootSchema as it was done before? Thanks

        Show
        jcamachorodriguez Jesus Camacho Rodriguez added a comment - - edited Julian Hyde , what do you think? Why did that method change, instead of using rootSchema as it was done before? Thanks
        Hide
        julianhyde Julian Hyde added a comment - - edited

        I changed the apply method because we were wanted to use the schema from Frameworks.PrepareAction.getConfig().getDefaultSchema() in preference to CalcitePrepare.Context.getRootSchema(). If you back out the change, RelBuilderTest fails to find any tables.

        But I didn't think of the case where FrameworkConfig.getDefaultSchema() returns null. The method javadoc says it is allowed to, and it seems that Hive is doing that.

        Can you try the following patch and see if it improves things:

        diff --git a/core/src/main/java/org/apache/calcite/tools/Frameworks.java b/core/src/main/java/org/apache/calcite/tools/Frameworks.java
        index c73a1e2..8f245f2 100644
        --- a/core/src/main/java/org/apache/calcite/tools/Frameworks.java
        +++ b/core/src/main/java/org/apache/calcite/tools/Frameworks.java
        @@ -36,6 +36,7 @@
         
         import com.google.common.base.Preconditions;
         import com.google.common.collect.ImmutableList;
        +import org.apache.calcite.util.Util;
         
         import java.sql.Connection;
         import java.sql.DriverManager;
        @@ -107,7 +108,8 @@ public abstract R apply(RelOptCluster cluster, RelOptSchema relOptSchema,
                   public R apply(RelOptCluster cluster, RelOptSchema relOptSchema,
                       SchemaPlus rootSchema, CalciteServerStatement statement) {
                     final CalciteSchema schema =
        -                CalciteSchema.from(config.getDefaultSchema());
        +                CalciteSchema.from(
        +                    Util.first(config.getDefaultSchema(), rootSchema));
                     return action.apply(cluster, relOptSchema, schema.root().plus());
                   }
                 });
        
        Show
        julianhyde Julian Hyde added a comment - - edited I changed the apply method because we were wanted to use the schema from Frameworks.PrepareAction.getConfig().getDefaultSchema() in preference to CalcitePrepare.Context.getRootSchema(). If you back out the change, RelBuilderTest fails to find any tables. But I didn't think of the case where FrameworkConfig.getDefaultSchema() returns null. The method javadoc says it is allowed to, and it seems that Hive is doing that. Can you try the following patch and see if it improves things: diff --git a/core/src/main/java/org/apache/calcite/tools/Frameworks.java b/core/src/main/java/org/apache/calcite/tools/Frameworks.java index c73a1e2..8f245f2 100644 --- a/core/src/main/java/org/apache/calcite/tools/Frameworks.java +++ b/core/src/main/java/org/apache/calcite/tools/Frameworks.java @@ -36,6 +36,7 @@ import com.google.common.base.Preconditions; import com.google.common.collect.ImmutableList; +import org.apache.calcite.util.Util; import java.sql.Connection; import java.sql.DriverManager; @@ -107,7 +108,8 @@ public abstract R apply(RelOptCluster cluster, RelOptSchema relOptSchema, public R apply(RelOptCluster cluster, RelOptSchema relOptSchema, SchemaPlus rootSchema, CalciteServerStatement statement) { final CalciteSchema schema = - CalciteSchema.from(config.getDefaultSchema()); + CalciteSchema.from( + Util.first(config.getDefaultSchema(), rootSchema)); return action.apply(cluster, relOptSchema, schema.root().plus()); } });
        Hide
        jcamachorodriguez Jesus Camacho Rodriguez added a comment -

        Julian Hyde, that solves the issue; FYI, there is a style problem in the code snippet related to the position of the new import.

        Show
        jcamachorodriguez Jesus Camacho Rodriguez added a comment - Julian Hyde , that solves the issue; FYI, there is a style problem in the code snippet related to the position of the new import .
        Hide
        julianhyde Julian Hyde added a comment -

        Testing a fix now.

        Show
        julianhyde Julian Hyde added a comment - Testing a fix now.
        Show
        julianhyde Julian Hyde added a comment - Fixed in http://git-wip-us.apache.org/repos/asf/incubator-calcite/commit/1580bd0f .
        Hide
        jnadeau Jacques Nadeau added a comment -

        Resolved in release 1.4.0-incubating (2015-08-23)

        Show
        jnadeau Jacques Nadeau added a comment - Resolved in release 1.4.0-incubating (2015-08-23)

          People

          • Assignee:
            julianhyde Julian Hyde
            Reporter:
            jcamachorodriguez Jesus Camacho Rodriguez
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development