[FLINK-25960] Distribute the data read buffers more fairly among result partitions for sort-shuffle

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.15.0
• Component/s: Runtime / Network
• Labels: pull-request-available
• Assignee: Yingjie Cao
• Reporter: Yingjie Cao
• Created: 04/Feb/22 11:32
• Updated: 09/Feb/22 11:41
• Resolved: 09/Feb/22 11:41

Description

Currently, the data read buffers for sort-shuffle are allocated in a random way and some result partitions may occupy too many buffers which leads to the starvation of other result partitions. This ticket aims to improve the scenario by not reading data for those result partitions which already occupy more than the average number of read buffers per result partition.

Issue Links

• links to: GitHub Pull Request #18631