Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-998

Exception when calling STDDEV_SAMP, STDDEV_POP

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.5.0
    • Fix Version/s: 1.7.0
    • Component/s: core
    • Labels:
      None

      Description

      Two incomplete but failing unit tests.

        @Test public void testStddevSampOnKnownType() throws Exception {
          CalciteAssert.that()
              .with(CalciteAssert.Config.FOODMART_CLONE)
              .query("SELECT STDDEV_SAMP(CAST(\"store_id\" AS INT)) AS col FROM \"store\"")
              .returns("");
        }
      
        @Test public void testStddevPopOnKnownType() throws Exception {
          CalciteAssert.that()
              .with(CalciteAssert.Config.FOODMART_CLONE)
              .query("SELECT STDDEV_POP(CAST(\"store_id\" AS INT)) AS col FROM \"store\"")
              .returns("");
        }
      

      Stack trace:

      java.lang.AssertionError: Internal error: Error while applying rule AggregateReduceFunctionsRule, args [rel#11:LogicalAggregate.NONE.[](input=rel#10:Subset#2.NONE.[0],group={},COL=STDDEV_SAMP($0))]
      
      	at org.apache.calcite.util.Util.newInternal(Util.java:792)
      	at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:251)
      	at org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:827)
      	at org.apache.calcite.tools.Programs$5.run(Programs.java:273)
      	at org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:321)
      	at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:145)
      	at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:282)
      	at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:190)
      	at org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:728)
      	at org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:587)
      	at org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:557)
      	at org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:214)
      	at org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:573)
      	at org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:571)
      	at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:135)
      	at org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:186)
      	at org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:469)
      	at org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1202)
      	at org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1185)
      	at org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1149)
      	at org.apache.calcite.test.JdbcTest.testStddevOnKnownType(JdbcTest.java:2885)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
      	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
      	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
      	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
      	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
      	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
      	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
      	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
      	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
      	at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
      	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:117)
      	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234)
      	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
      Caused by: java.lang.IndexOutOfBoundsException: index (1) must be less than size (1)
      	at com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:305)
      	at com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:284)
      	at com.google.common.collect.SingletonImmutableList.get(SingletonImmutableList.java:45)
      	at org.apache.calcite.sql.type.SqlTypeUtil$1.get(SqlTypeUtil.java:1132)
      	at org.apache.calcite.sql.type.SqlTypeUtil$1.get(SqlTypeUtil.java:1130)
      	at org.apache.calcite.rel.core.Aggregate$AggCallBinding.getOperandType(Aggregate.java:506)
      	at org.apache.calcite.sql.type.ReturnTypes$13.inferReturnType(ReturnTypes.java:743)
      	at org.apache.calcite.sql.SqlOperator.inferReturnType(SqlOperator.java:458)
      	at org.apache.calcite.rel.core.AggregateCall.create(AggregateCall.java:120)
      	at org.apache.calcite.rel.rules.AggregateReduceFunctionsRule.reduceStddev(AggregateReduceFunctionsRule.java:391)
      	at org.apache.calcite.rel.rules.AggregateReduceFunctionsRule.reduceAgg(AggregateReduceFunctionsRule.java:211)
      	at org.apache.calcite.rel.rules.AggregateReduceFunctionsRule.reduceAggs(AggregateReduceFunctionsRule.java:163)
      	at org.apache.calcite.rel.rules.AggregateReduceFunctionsRule.onMatch(AggregateReduceFunctionsRule.java:107)
      	at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:228)
      	... 45 more
      

      These are most likely regressions introduced by CALCITE-845 (this commit).

        Activity

        Hide
        julianhyde Julian Hyde added a comment -

        Resolved in release 1.7.0 (2016-03-22).

        Show
        julianhyde Julian Hyde added a comment - Resolved in release 1.7.0 (2016-03-22).
        Hide
        julianhyde Julian Hyde added a comment - - edited

        Fixed in http://git-wip-us.apache.org/repos/asf/calcite/commit/7ce2e621. Thanks for the PR, Matthew Shaer!

        I refactored your pull request a bit. I didn't want complicated stuff going on inside AggregateCall.create, so I pulled the type-derivation outside. But you were spot-on as to the root cause - operand ordinals being in terms of a RelNode that had not yet been created. I also added a couple of tests, and enabled some old ones that had never been enabled before.

        Show
        julianhyde Julian Hyde added a comment - - edited Fixed in http://git-wip-us.apache.org/repos/asf/calcite/commit/7ce2e621 . Thanks for the PR, Matthew Shaer ! I refactored your pull request a bit. I didn't want complicated stuff going on inside AggregateCall.create, so I pulled the type-derivation outside. But you were spot-on as to the root cause - operand ordinals being in terms of a RelNode that had not yet been created. I also added a couple of tests, and enabled some old ones that had never been enabled before.
        Hide
        mshaer@gmail.com Matthew Shaer added a comment -

        It does now

        Show
        mshaer@gmail.com Matthew Shaer added a comment - It does now
        Hide
        julianhyde Julian Hyde added a comment -

        Does mvn install complete successfully after your changes?

        Show
        julianhyde Julian Hyde added a comment - Does mvn install complete successfully after your changes?
        Hide
        mshaer@gmail.com Matthew Shaer added a comment - - edited

        Created pull request https://github.com/apache/calcite/pull/191 to fix

        Show
        mshaer@gmail.com Matthew Shaer added a comment - - edited Created pull request https://github.com/apache/calcite/pull/191 to fix
        Hide
        julianhyde Julian Hyde added a comment -

        I can reproduce it also. Here is a simpler patch to reproduce the issue.

        diff --git a/core/src/test/resources/sql/agg.oq b/core/src/test/resources/sql/agg.oq
        index e92f08d..78bcb82 100644
        --- a/core/src/test/resources/sql/agg.oq
        +++ b/core/src/test/resources/sql/agg.oq
        @@ -62,6 +62,19 @@ select count(deptno, ename, 1, deptno) as c from emp;
         
         !ok
         
        +# [CALCITE-998] Query with stddev fails with internal error
        +# stddev_samp
        +select stddev_samp(deptno) as s from emp;
        +!ok
        +
        +# stddev_pop
        +select stddev_pop(deptno) as s from emp;
        +!ok
        +
        +# stddev
        +select stddev(deptno) as s from emp;
        +!ok
        +
         select city, gender as c from emps;
         +---------------+---+
         | CITY          | C |
        
        Show
        julianhyde Julian Hyde added a comment - I can reproduce it also. Here is a simpler patch to reproduce the issue. diff --git a/core/src/test/resources/sql/agg.oq b/core/src/test/resources/sql/agg.oq index e92f08d..78bcb82 100644 --- a/core/src/test/resources/sql/agg.oq +++ b/core/src/test/resources/sql/agg.oq @@ -62,6 +62,19 @@ select count(deptno, ename, 1, deptno) as c from emp; !ok +# [CALCITE-998] Query with stddev fails with internal error +# stddev_samp +select stddev_samp(deptno) as s from emp; +!ok + +# stddev_pop +select stddev_pop(deptno) as s from emp; +!ok + +# stddev +select stddev(deptno) as s from emp; +!ok + select city, gender as c from emps; +---------------+---+ | CITY | C |

          People

          • Assignee:
            julianhyde Julian Hyde
            Reporter:
            sudheeshkatkam Sudheesh Katkam
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development