[FLINK-21790] Shuffle data directories to make directory selection of different TaskManagers fairer

• Type: Sub-task
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 1.15.0
• Component/s: Runtime / Network
• Labels: pull-request-available
• Assignee: Yuxin Tan
• Reporter: Yingjie Cao
• Created: 15/Mar/21 06:37
• Updated: 26/Jan/22 02:08
• Resolved: 26/Jan/22 02:08

Description

Currently, different TaskManagers select data directory in the same order and if there are multiple disk, some disks may stores more data than others which is bad for performance. A simple improvement is that each TaskManager shuffles the given data directories randomly and select the data directory in different order.

Issue Links

• links to: GitHub Pull Request #18456