Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-1852

cube routing problem with distinct count

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • v1.5.2
    • None
    • Query Engine
    • CentOS release 6.6

    Description

      问题描述：
      同一个model 建立两个cube, 在这称为cube1 与 cube2. 两个cube有相同的measure, 除了cube2多一个distinct_count measure , 这儿称为
      a 。
      查询时使用 select count(distinct a) from table1. 这个时候查询出错，提示Can't create EnumerableAggregate! 日志中查询选择的cube
      为cube1 而不是cube2。 以下为详细日志：

      2016-07-05 21:18:57,839 INFO [http-bio-7070-exec-2] schema.OLAPSchemaFactory:135 : Schema json:{
      "version": "1.0",
      "defaultSchema": "DEFAULT",
      "schemas": [
      {
      "type": "custom",
      "name": "DEFAULT",
      "factory": "org.apache.kylin.query.schema.OLAPSchemaFactory",
      "operand":

      { "project": "TEST" }

      ,
      "functions": [

      { name: 'MASSIN', className: 'org.apache.kylin.query.udf.MassInUDF' }

      ]
      }
      ]
      }
      2016-07-05 21:18:58,701 DEBUG [http-bio-7070-exec-2] service.QueryService:288 : getting table metas
      2016-07-05 21:18:58,817 DEBUG [http-bio-7070-exec-2] service.QueryService:306 : getting column metas
      2016-07-05 21:18:58,920 DEBUG [http-bio-7070-exec-2] service.QueryService:320 : done column metas
      2016-07-05 21:19:18,991 INFO [http-bio-7070-exec-5] controller.QueryController:174 : Using project: test
      2016-07-05 21:19:18,995 INFO [http-bio-7070-exec-5] controller.QueryController:175 : The original query: select count(DISTINCT order_title) from hwb_alipay_trade
      2016-07-05 21:19:19,009 INFO [http-bio-7070-exec-5] service.QueryService:264 : The corrected query: select count(DISTINCT order_title) from hwb_alipay_trade
      LIMIT 50000
      2016-07-05 21:19:19,968 INFO [http-bio-7070-exec-5] routing.QueryRouter:48 : The project manager's reference is org.apache.kylin.metadata.project.ProjectManager@7fab0bdb
      2016-07-05 21:19:19,970 INFO [http-bio-7070-exec-5] routing.QueryRouter:60 : Find candidates by table DEFAULT.HWB_ALIPAY_TRADE and project=TEST : org.apache.kylin.query.routing.Candidate@5a6373b7,org.apache.kylin.query.routing.Candidate@7db14ef3
      2016-07-05 21:19:19,974 INFO [http-bio-7070-exec-5] routing.QueryRouter:49 : Applying rule: class org.apache.kylin.query.routing.rules.RemoveUncapableRealizationsRule, realizations before: [hwb_alipay_cube_distinct(CUBE),hwb_alipay_cube(CUBE)], realizations after: [hwb_alipay_cube_distinct(CUBE),hwb_alipay_cube(CUBE)]
      2016-07-05 21:19:19,975 INFO [http-bio-7070-exec-5] routing.QueryRouter:49 : Applying rule: class org.apache.kylin.query.routing.rules.RealizationSortRule, realizations before: [hwb_alipay_cube_distinct(CUBE),hwb_alipay_cube(CUBE)], realizations after: [hwb_alipay_cube(CUBE),hwb_alipay_cube_distinct(CUBE)]
      2016-07-05 21:19:19,975 INFO [http-bio-7070-exec-5] routing.QueryRouter:85 : Adjust DimensionAsMeasure for FunctionDesc [expression=COUNT_DISTINCT, parameter=ParameterDesc [type=column, value=ORDER_TITLE, nextParam=null], returnType=null]
      2016-07-05 21:19:19,976 INFO [http-bio-7070-exec-5] routing.QueryRouter:72 : The realizations remaining: [hwb_alipay_cube(CUBE),hwb_alipay_cube_distinct(CUBE)] And the final chosen one is the first one
      2016-07-05 21:19:19,987 ERROR [http-bio-7070-exec-5] controller.QueryController:209 : Exception when execute sql
      java.sql.SQLException: Error while executing SQL "select count(DISTINCT order_title) from hwb_alipay_trade
      LIMIT 50000": Can't create EnumerableAggregate!
      at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
      at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
      at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:143)
      at org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:186)
      at org.apache.kylin.rest.service.QueryService.execute(QueryService.java:355)
      at org.apache.kylin.rest.service.QueryService.queryWithSqlMassage(QueryService.java:271)
      at org.apache.kylin.rest.service.QueryService.query(QueryService.java:119)
      at org.apache.kylin.rest.service.QueryService$$FastClassByCGLIB$$4957273f.invoke(<generated>)
      at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
      at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:618)
      at org.apache.kylin.rest.service.QueryService$$EnhancerByCGLIB$$4398392b.query(<generated>)
      at org.apache.kylin.rest.controller.QueryController.doQueryWithCache(QueryController.java:192)
      at org.apache.kylin.rest.controller.QueryController.query(QueryController.java:94)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:497)
      at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:213)
      at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:126)
      at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:96)
      at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:617)
      at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:578)
      at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80)
      at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:923)
      at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:852)
      at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:882)
      at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:789)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:646)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
      at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
      at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
      at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
      at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
      at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
      at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
      at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
      at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103)
      at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
      at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
      at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
      at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:54)
      at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
      at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
      at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
      at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:150)
      at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
      at org.springframework.security.web.authentication.ui.DefaultLoginPageGeneratingFilter.doFilter(DefaultLoginPageGeneratingFilter.java:91)
      at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
      at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:183)
      at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
      at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:105)
      at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
      at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
      at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
      at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)
      at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
      at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
      at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
      at com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:195)
      at com.thetransactioncompany.cors.CORSFilter.doFilter(CORSFilter.java:266)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
      at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220)
      at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)
      at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504)
      at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:170)
      at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
      at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:950)
      at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
      at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:421)
      at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1074)
      at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:611)
      at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:314)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
      at java.lang.Thread.run(Thread.java:745)
      Caused by: java.lang.IllegalStateException: Can't create EnumerableAggregate!
      at org.apache.kylin.query.relnode.OLAPAggregateRel.implementEnumerable(OLAPAggregateRel.java:371)
      at org.apache.kylin.query.relnode.OLAPRel$JavaImplementor.createEnumerable(OLAPRel.java:159)
      at org.apache.kylin.query.relnode.OLAPRel$JavaImplementor.createEnumerable(OLAPRel.java:155)
      at org.apache.kylin.query.relnode.OLAPToEnumerableConverter.implement(OLAPToEnumerableConverter.java:99)
      at org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.implementRoot(EnumerableRelImplementor.java:102)
      at org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:92)
      at org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1171)
      at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:297)
      at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:196)
      at org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:721)
      at org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:588)
      at org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:558)
      at org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:214)
      at org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:573)
      at org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:571)
      at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:135)
      ... 80 more
      Caused by: org.apache.calcite.rel.InvalidRelException: distinct aggregation not supported
      at org.apache.calcite.adapter.enumerable.EnumerableAggregate.<init>(EnumerableAggregate.java:71)
      at org.apache.kylin.query.relnode.OLAPAggregateRel.implementEnumerable(OLAPAggregateRel.java:368)
      ... 95 more
      2016-07-05 21:19:19,993 INFO [http-bio-7070-exec-5] service.QueryService:248 :
      ==========================[QUERY]===============================
      SQL: select count(DISTINCT order_title) from hwb_alipay_trade
      User: ADMIN
      Success: false
      Duration: 0.0
      Project: test
      Realization Names: [hwb_alipay_cube]
      Cuboid Ids: []
      Total scan count: 0
      Result row count: 0
      Accept Partial: true
      Is Partial Result: false
      Hit Exception Cache: false
      Storage cache used: false
      Message: Error while executing SQL "select count(DISTINCT order_title) from hwb_alipay_trade LIMIT 50000": Can't create EnumerableAggregate!

      Attachments

        Activity

          People

            liyang.gmt8@gmail.com liyang
            binghuang bing huang
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: