[FLINK-30081] Local executor can not accept different jvm-overhead.min/max values

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.16.0
• Fix Version/s: 1.20.0
• Component/s: Runtime / Configuration
• Labels: auto-deprioritized-major pull-request-available
• Assignee: Mingliang Liu
• Reporter: Mingliang Liu
• Created: 18/Nov/22 06:30
• Updated: 16/Apr/24 06:25
• Resolved: 16/Apr/24 06:25

Description

In local executor, it's not possible to set different values for taskmanager.memory.jvm-overhead.max and taskmanager.memory.jvm-overhead.min. The same problem for taskmanager.memory.network.max and taskmanager.memory.network.min.

Sample code to reproduce:

Configuration conf = new Configuration();
conf.setString(TaskManagerOptions.JVM_OVERHEAD_MIN.key(), "1GB");
conf.setString(TaskManagerOptions.JVM_OVERHEAD_MAX.key(), "2GB");

StreamExecutionEnvironment.createLocalEnvironment(conf)
    .fromElements("Hello", "World")
    .executeAndCollect()
    .forEachRemaining(System.out::println); 

The failing exception is something like:

Exception in thread "main" java.lang.IllegalArgumentException
  at org.apache.flink.util.Preconditions.checkArgument(Preconditions.java:122)
  at org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils.calculateTotalProcessMemoryFromComponents(TaskExecutorResourceUtils.java:182)
  at org.apache.flink.runtime.taskexecutor.TaskExecutorMemoryConfiguration.create(TaskExecutorMemoryConfiguration.java:119)

I think the problem was that we expect the max and min to equal, but local executor did not reset them correctly?

Issue Links

• links to: GitHub Pull Request #22547