[FLINK-25636] FLIP-199: Change some default config values of blocking shuffle for better usability

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.15.0
• Component/s: Runtime / Network
• Labels: pull-request-available

Release Note

Since 1.15, sort-shuffle has become the default blocking shuffle implementation and shuffle data compression is enabled by default. These changes only influence batch jobs and can improve the blocking shuffle performance and stability. For more information, please refer to the official document: https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/ops/batch/blocking_shuffle.

• Assignee: Yingjie Cao
• Reporter: Yingjie Cao
• Created: 13/Jan/22 06:21
• Updated: 29/Apr/22 06:32
• Resolved: 29/Apr/22 04:55

Description

This is the umbrella issue for FLIP-199, we will change the several default config value for batch shuffle and update the document accordingly.

Issue Links

• relates to: FLINK-24898 Some further improvements of sort-shuffle
• links to: GitHub Pull Request #18350

Sub-Tasks

• Enable data compression for blocking shuffle by default: Closed
• Make sort-shuffle the default shuffle implementation for batch jobs: Closed
• Increase the default write buffer size of sort-shuffle to 16M: Closed
• Increase the default read buffer size of sort-shuffle to 64M: Closed
• Enhance the document for blocking shuffle: Closed