Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-35461

Deprecate Some Runtime Configuration for Flink 2.0

    XMLWordPrintableJSON

Details

    • Hide
      The following configurations have been deprecated as we are phasing out the hash-based blocking shuffle:
      - `taskmanager.network.sort-shuffle.min-parallelism`
      - `taskmanager.network.blocking-shuffle.type`

      The following configurations have been deprecated as we are phasing out the legacy hybrid shuffle:
      - `taskmanager.network.hybrid-shuffle.spill-index-region-group-size`
      - `taskmanager.network.hybrid-shuffle.num-retained-in-memory-regions-max`
      - `taskmanager.network.hybrid-shuffle.enable-new-mode`

      The following configurations have been deprecated to simply the configuration of network buffers:
      - `taskmanager.network.memory.buffers-per-channel`
      - `taskmanager.network.memory.floating-buffers-per-gate`
      - `taskmanager.network.memory.max-buffers-per-channel`
      - `taskmanager.network.memory.max-overdraft-buffers-per-gate`
      - `taskmanager.network.memory.exclusive-buffers-request-timeout-ms` (Please use `taskmanager.network.memory.buffers-request-timeout` instead.)

      The configuration `taskmanager.network.batch-shuffle.compression.enabled` has been deprecated. Please set `taskmanager.network.compression.codec` to "NONE" to disable compression.

      The following Netty-related configurations are no longer recommended for use and have been deprecated:
      - `taskmanager.network.netty.num-arenas`
      - `taskmanager.network.netty.server.numThreads`
      - `taskmanager.network.netty.client.numThreads`
      - `taskmanager.network.netty.server.backlog`
      - `taskmanager.network.netty.sendReceiveBufferSize`
      - `taskmanager.network.netty.transport`

      The following configurations are unnecessary and have been deprecated:
      - `taskmanager.network.max-num-tcp-connections`
      - `fine-grained.shuffle-mode.all-blocking`
      Show
      The following configurations have been deprecated as we are phasing out the hash-based blocking shuffle: - `taskmanager.network.sort-shuffle.min-parallelism` - `taskmanager.network.blocking-shuffle.type` The following configurations have been deprecated as we are phasing out the legacy hybrid shuffle: - `taskmanager.network.hybrid-shuffle.spill-index-region-group-size` - `taskmanager.network.hybrid-shuffle.num-retained-in-memory-regions-max` - `taskmanager.network.hybrid-shuffle.enable-new-mode` The following configurations have been deprecated to simply the configuration of network buffers: - `taskmanager.network.memory.buffers-per-channel` - `taskmanager.network.memory.floating-buffers-per-gate` - `taskmanager.network.memory.max-buffers-per-channel` - `taskmanager.network.memory.max-overdraft-buffers-per-gate` - `taskmanager.network.memory.exclusive-buffers-request-timeout-ms` (Please use `taskmanager.network.memory.buffers-request-timeout` instead.) The configuration `taskmanager.network.batch-shuffle.compression.enabled` has been deprecated. Please set `taskmanager.network.compression.codec` to "NONE" to disable compression. The following Netty-related configurations are no longer recommended for use and have been deprecated: - `taskmanager.network.netty.num-arenas` - `taskmanager.network.netty.server.numThreads` - `taskmanager.network.netty.client.numThreads` - `taskmanager.network.netty.server.backlog` - `taskmanager.network.netty.sendReceiveBufferSize` - `taskmanager.network.netty.transport` The following configurations are unnecessary and have been deprecated: - `taskmanager.network.max-num-tcp-connections` - `fine-grained.shuffle-mode.all-blocking`

    Description

      As Flink moves toward 2.0, we have revisited all runtime configurations and identified several improvements to enhance user-friendliness and maintainability. We want to refine the runtime configuration.

       

      This ticket implements all the changes discussed in FLIP-450.

      Attachments

        Issue Links

          Activity

            People

              sxnan Xuannan Su
              sxnan Xuannan Su
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: