[FLINK-25780] Reduce the maximum size of data output buffers per result partition for sort-shuffle - ASF JIRA

Attach files

Attach Screenshot

Voters

Watch issue

Watchers

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Sub-task
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 1.15.0
Component/s: Runtime / Network
Labels:
- pull-request-available

Description

The data output buffer of sort-shuffle is for better disk IO performance and currently, the total data output buffer size is 16M which is pretty big. However, blocking request too many buffers may influence performance. This ticket aims to reduce the maximum number of data output buffers to reduce the buffer request time. The selected value is an empirical one based on the TPC-DS test results.