[IMPALA-10229] Analytic limit pushdown optimization can be applied incorrectly based on predicates present

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: Impala 4.0.0
• Component/s: Frontend
• Labels: correctness
• Target Version: Impala 4.0.0
• Epic Color: ghx-label-12
• Assignee: Tim Armstrong
• Reporter: Tim Armstrong
• Created: 08/Oct/20 21:49
• Updated: 25/Jan/21 19:00
• Resolved: 25/Jan/21 19:00

Description

[localhost.EXAMPLE.COM:21050] default> select * from (select month, id, rank() over (partition by month order by id desc) rnk from functional_parquet.alltypes WHERE month >= 11) v order by month, id limit 3;
+-------+------+-----+
| month | id   | rnk |
+-------+------+-----+
| 11    | 6987 | 3   |
| 11    | 6988 | 2   |
| 11    | 6989 | 1   |
+-------+------+-----+
Fetched 3 row(s) in 4.16s

These are not the top 3 rows when ordering by month, id . Hive's result is correct:

+----------+-------+--------+
| v.month  | v.id  | v.rnk  |
+----------+-------+--------+
| 11       | 3040  | 600    |
| 11       | 3041  | 599    |
| 11       | 3042  | 598    |
+----------+-------+--------+

I think when there's no select predicates, that the ordering in the analytic sort needs to exactly match the TOP N sort ordering. I'm not sure if there are fixes needed for the case where there are select predicates.

Sub-Tasks

• Fix analytic limit pushdown when no predicates are present: Resolved
• Fix analytic limit pushdown when predicates are present: Resolved