[FLINK-35461] Deprecate Some Runtime Configuration for Flink 2.0

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Done
• Fix Version/s: 1.20.0
• Component/s: Runtime / Configuration
• Labels: pull-request-available

Release Note

The following configurations have been deprecated as we are phasing out the hash-based blocking shuffle:
- `taskmanager.network.sort-shuffle.min-parallelism`
- `taskmanager.network.blocking-shuffle.type`

The following configurations have been deprecated as we are phasing out the legacy hybrid shuffle:
- `taskmanager.network.hybrid-shuffle.spill-index-region-group-size`
- `taskmanager.network.hybrid-shuffle.num-retained-in-memory-regions-max`
- `taskmanager.network.hybrid-shuffle.enable-new-mode`

The following configurations have been deprecated to simply the configuration of network buffers:
- `taskmanager.network.memory.buffers-per-channel`
- `taskmanager.network.memory.floating-buffers-per-gate`
- `taskmanager.network.memory.max-buffers-per-channel`
- `taskmanager.network.memory.max-overdraft-buffers-per-gate`
- `taskmanager.network.memory.exclusive-buffers-request-timeout-ms` (Please use `taskmanager.network.memory.buffers-request-timeout` instead.)

The configuration `taskmanager.network.batch-shuffle.compression.enabled` has been deprecated. Please set `taskmanager.network.compression.codec` to "NONE" to disable compression.

The following Netty-related configurations are no longer recommended for use and have been deprecated:
- `taskmanager.network.netty.num-arenas`
- `taskmanager.network.netty.server.numThreads`
- `taskmanager.network.netty.client.numThreads`
- `taskmanager.network.netty.server.backlog`
- `taskmanager.network.netty.sendReceiveBufferSize`
- `taskmanager.network.netty.transport`

The following configurations are unnecessary and have been deprecated:
- `taskmanager.network.max-num-tcp-connections`
- `fine-grained.shuffle-mode.all-blocking`

• Assignee: Xuannan Su
• Reporter: Xuannan Su
• Created: 27/May/24 03:48
• Updated: 08/Jul/24 09:20
• Resolved: 30/May/24 07:01

Description

As Flink moves toward 2.0, we have revisited all runtime configurations and identified several improvements to enhance user-friendliness and maintainability. We want to refine the runtime configuration.

 

This ticket implements all the changes discussed in FLIP-450.

Issue Links

• links to: GitHub Pull Request #24853
• mentioned in: Page Loading...