[FLINK-21054] Implement mini-batch optimized slicing window aggregate operator

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.13.0
• Component/s: Table SQL / Runtime
• Labels: pull-request-available
• Assignee: Jark Wu
• Reporter: Jark Wu
• Created: 20/Jan/21 10:16
• Updated: 28/May/21 07:13
• Resolved: 25/Jan/21 08:06

Description

We have supported cumulative windows in FLINK-19605. However, the current cumulative window is not efficient, because the slices are not shared.

We leverages the slicing ideas proposed in FLINK-7001 and this design doc [1]. The slicing is an optimized implementation for hopping, cumulative, tumbling windows. Besides of that, we introduced ManagedMemory based mini-batch optimization for the slicing window aggregate operator, this can tremendously reduce the accessing of state and get the higher throughtput without latency loss.

[1]: https://docs.google.com/document/d/1ziVsuW_HQnvJr_4a9yKwx_LEnhVkdlde2Z5l6sx5HlY/edit#

Issue Links

• links to: GitHub Pull Request #14708