[FLINK-19896] Improve first-n-row fetching in the rank operator

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.14.0
• Component/s: Table SQL / Runtime
• Labels: auto-deprioritized-major pull-request-available
• Assignee: Jun Zhang
• Reporter: Jun Zhang
• Created: 30/Oct/20 09:00
• Updated: 28/Aug/21 12:14
• Resolved: 03/Jun/21 02:21

Description

Currently Deduplicate operator only supports first-row deduplication (ordered by proc-time). In scenario of first-n-rows deduplication, the planner has to resort to Rank operator. However, Rank operator is less efficient than Deduplicate due to larger state and more state access.

This issue proposes to extend DeduplicateKeepFirstRowFunction to support first-n-rows deduplication. And the original first-row deduplication would be a special case of first-n-rows deduplication.

Issue Links

• links to: GitHub Pull Request #13921