[FLINK-25786] Adjust the generation of subpartition data storage order for sort-shuffle from random shuffle to random shift

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.15.0
• Component/s: Runtime / Network
• Labels: pull-request-available
• Assignee: Yingjie Cao
• Reporter: Yingjie Cao
• Created: 24/Jan/22 13:06
• Updated: 27/Jan/22 08:05
• Resolved: 27/Jan/22 08:05

Description

Currently, for sort-shuffle the generation of subpartition data storage order  is random shuffle. However, if there is no enough resources to run the downstream consumer tasks in parallel, the performance can be influenced because of the random disk IO caused by the random subpartition data storage order. This ticket aims to improve this scenario.

Issue Links

• links to: GitHub Pull Request #18474