Details
-
New Feature
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
v4.0.0-alpha
-
None
Description
when use top-level result offset clause in query expression (ANSI SQL) : limit xxx offset xxx in spark query engine,limit will not push down into spark engine, and offset will not take effect. This is incompatible wIth Kylin 2.x~3.x.
After looking through the code, i found it's because spark dose not support limit ... offset ... now. There is a spark issue in progress: https://issues.apache.org/jira/browse/SPARK-28330, which was created in 2019 but still in progress.
So, should we support this feature temporarily in KYLIN? :
1. push down limit to spark
2. take result from starting offset in KYLIN query server