[KYLIN-3091] A problem about retention rate analyze

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Resolved
• Affects Version/s: v2.0.0
• Fix Version/s: v2.2.0
• Component/s: Query Engine
• Environment: hbase 0.98.8-hadoop2
• Assignee: Kaisen Kang
• Reporter: Sheng Wang
• Created: 08/Dec/17 10:24
• Updated: 15/Dec/17 12:33
• Resolved: 15/Dec/17 12:33

Description

I found that kylin supported retention rate analyze function, so I made some test for this function. The following SQL executed successful:

select city, version,
intersect_count(uuid, dt, array['20161014', '20161015']) as retention_oneday,
intersect_count(uuid, dt, array['20161014', '20161015', '20161016']) as retention_twoday
from visit_log
where dt in ('2016104', '20161015', '20161016')
group by city, version

but, other SQLs executed failed like this:

select city,
intersect_count(uuid, dt, array['20161014', '20161015']) as retention_oneday
from visit_log 
where dt in ('2016104', '20161015',) 
group by city, version

select city, version,
intersect_count(uuid, dt, array['20161014', '20161015', '20161016']) as retention_twoday
from visit_log 
where dt in ('2016104', '20161015', '20161016') 
group by city, version

which means I cannot use just one intersect_count UDAF in a SQL, at lease two intersect_count. My kylin version is kylin 2.0.0-hbase 0.98.8, and here is the error log:

Caused by: java.lang.IndexOutOfBoundsException: Index: 2, Size: 2
        at java.util.ArrayList.rangeCheck(ArrayList.java:635)
        at java.util.ArrayList.get(ArrayList.java:411)
        at org.apache.kylin.query.relnode.ColumnRowType.getColumnByIndex(ColumnRowType.java:49)
        at org.apache.kylin.query.relnode.OLAPAggregateRel.fillbackOptimizedColumn(OLAPAggregateRel.java:396)
        at org.apache.kylin.query.relnode.OLAPAggregateRel.buildRewriteFieldsAndMetricsColumns(OLAPAggregateRel.java:347)
        at org.apache.kylin.query.relnode.OLAPAggregateRel.implementRewrite(OLAPAggregateRel.java:283)
        at org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:158)
        at org.apache.kylin.query.relnode.OLAPLimitRel.implementRewrite(OLAPLimitRel.java:107)
        at org.apache.kylin.query.relnode.OLAPRel$RewriteImplementor.visitChild(OLAPRel.java:158)
        at org.apache.kylin.query.relnode.OLAPToEnumerableConverter.implement(OLAPToEnumerableConverter.java:100)
        at org.apache.calcite.adapter.enumerable.EnumerableRelImplementor.implementRoot(EnumerableRelImplementor.java:108)
        at org.apache.calcite.adapter.enumerable.EnumerableInterpretable.toBindable(EnumerableInterpretable.java:92)
        at org.apache.calcite.prepare.CalcitePrepareImpl$CalcitePreparingStmt.implement(CalcitePrepareImpl.java:1248)
        at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:306)
        at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:203)
        at org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:776)
        at org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:632)
        at org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:602)
        at org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:214)
        at org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:595)
        at org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:615)
        at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:148)