[FLINK-28823] Enlarge the max requested buffers for SortMergeResultPartitionReadScheduler

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.16.0
• Fix Version/s: 1.16.0
• Component/s: Runtime / Network
• Labels: pull-request-available
• Assignee: Yuxin Tan
• Reporter: Yuxin Tan
• Created: 05/Aug/22 06:33
• Updated: 08/Aug/22 07:27
• Resolved: 08/Aug/22 07:27

Description

The num of the max requested buffers in SortMergeResultPartitionReadScheduler is determined by the num of subpartitions, which may be insufficient in some scenarios, such as speculative execution and partition reuse.

Therefore, we should increase the max requested buffers to make full use of available memory when TM memory is sufficient.

 

Issue Links

• links to: GitHub Pull Request #20459