[KYLIN-4853] QueryPreparedStatementCache invalid in Spark Query Engine

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: v4.0.0-beta
• Assignee: Shengjun Zheng
• Reporter: Shengjun Zheng
• Created: 31/Dec/20 10:38
• Updated: 05/Jan/21 10:34
• Resolved: 05/Jan/21 10:34

Description

Prepare statement cache  imported from  KYLIN-3434 get invalid in SparkQuery Engine.

In SparkQuery Engine, physical plan will be created directly from RelNode, which was not cached, it will cause  prepared statment query fail after first successful query.

Error Stack like this:

ava.sql.SQLException: exception while executing query: java.lang.NullPointerException
        at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
        at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
        at org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:577)
        at org.apache.calcite.avatica.AvaticaPreparedStatement.executeQuery(AvaticaPreparedStatement.java:137)
        at org.apache.kylin.rest.service.QueryService.executePrepareRequest(QueryService.java:1032)
        at org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:718)
Caused by: java.lang.RuntimeException: java.lang.NullPointerException
        at org.apache.kylin.query.exec.SparkExec.collectToEnumerable(SparkExec.java:46)
        at Baz.bind(Unknown Source)
        at org.apache.calcite.jdbc.CalcitePrepare$CalciteSignature.enumerable(CalcitePrepare.java:365)
        at org.apache.calcite.jdbc.CalciteConnectionImpl.enumerable(CalciteConnectionImpl.java:301)
        at org.apache.calcite.jdbc.CalciteMetaImpl._createIterable(CalciteMetaImpl.java:559)
        at org.apache.calcite.jdbc.CalciteMetaImpl.createIterable(CalciteMetaImpl.java:550)
        at org.apache.calcite.avatica.AvaticaResultSet.execute(AvaticaResultSet.java:182)
        at org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:67)
        at org.apache.calcite.jdbc.CalciteResultSet.execute(CalciteResultSet.java:44)
        at org.apache.calcite.avatica.AvaticaConnection.executeQueryInternal(AvaticaConnection.java:573)
        ... 84 more
Caused by: java.lang.NullPointerException
        at org.apache.kylin.query.runtime.CalciteToSparkPlaner.visit(CalciteToSparkPlaner.scala:45)
        at org.apache.calcite.rel.RelVisitor.go(RelVisitor.java:61)
        at org.apache.kylin.query.runtime.SparkEngine.toSparkPlan(SparkEngine.java:61)
        at org.apache.kylin.query.runtime.SparkEngine.compute(SparkEngine.java:49)
        at org.apache.kylin.query.exec.QueryEngineFactory.compute(QueryEngineFactory.java:47)
        at org.apache.kylin.query.exec.SparkExec.collectToEnumerable(SparkExec.java:43)

 

I  solved this by cache RelNode to avoid SQL be parsed, optimized, code gen again. wonder if there is better way. thanks ~

 

Issue Links

• links to: Patch