[FLINK-17970] Increase default value of IO pool executor to 4 * #cores

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.11.0
• Fix Version/s: 1.10.2, 1.11.0
• Component/s: Runtime / Coordination
• Labels: pull-request-available
• Assignee: Till Rohrmann
• Reporter: Till Rohrmann
• Created: 27/May/20 12:31
• Updated: 02/Jun/20 14:41
• Resolved: 02/Jun/20 14:41

Description

Currently, the default value of ClusterOptions.CLUSTER_IO_EXECUTOR_POOL_SIZE is #cores. I propose to increase it to 4 * #cores to support a higher load of blocking IO operations.

Moreover, I propose to use a cached thread pool instead of a fixed thread pool. That way, only those use cases which have high IO load will actually occupy the required resources to start more threads.

Last but not least, I propose to change the config option name from cluster.io-executor.pool-size to cluster.io-pool.size which is a bit shorter.

Issue Links

• links to: GitHub Pull Request #12360