[FLINK-17503] Make memory configuration logging more user-friendly

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.10.0
• Fix Version/s: 1.10.2, 1.11.2, 1.12.0
• Component/s: Runtime / Coordination
• Labels: pull-request-available usability
• Assignee: Matthias Pohl
• Reporter: Till Rohrmann
• Created: 04/May/20 11:52
• Updated: 08/Aug/20 10:17
• Resolved: 08/Aug/20 10:17

Description

The newly introduced memory configuration logs some output when using the Mini Cluster (or local environment):

2020-05-04 11:50:05,984 INFO  org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils [] - The configuration option Key: 'taskmanager.cpu.cores' , default: null (fallback keys: []) required for local execution is not set, setting it to its default value 1.7976931348623157E308
2020-05-04 11:50:05,989 INFO  org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils [] - The configuration option Key: 'taskmanager.memory.task.heap.size' , default: null (fallback keys: []) required for local execution is not set, setting it to its default value 9223372036854775807 bytes
2020-05-04 11:50:05,989 INFO  org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils [] - The configuration option Key: 'taskmanager.memory.task.off-heap.size' , default: 0 bytes (fallback keys: []) required for local execution is not set, setting it to its default value 9223372036854775807 bytes
2020-05-04 11:50:05,990 INFO  org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils [] - The configuration option Key: 'taskmanager.memory.network.min' , default: 64 mb (fallback keys: [{key=taskmanager.network.memory.min, isDeprecated=true}]) required for local execution is not set, setting it to its default value 64 mb
2020-05-04 11:50:05,990 INFO  org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils [] - The configuration option Key: 'taskmanager.memory.network.max' , default: 1 gb (fallback keys: [{key=taskmanager.network.memory.max, isDeprecated=true}]) required for local execution is not set, setting it to its default value 64 mb
2020-05-04 11:50:05,991 INFO  org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils [] - The configuration option Key: 'taskmanager.memory.managed.size' , default: null (fallback keys: [{key=taskmanager.memory.size, isDeprecated=true}]) required for local execution is not set, setting it to its default value 128 mb

This logging output could be made more user-friendly the following way:

• Print only the key string of a ConfigOption, not the config option object with all the deprecated keys
• Skipping the lines for taskmanager.memory.task.heap.size and taskmanager.memory.task.off-heap.size - we don't really set them (they are JVM paramaters) and the printing of long max looks strange (user would have to know these are place holders without effect).
• Maybe similarly skipping the CPU cores value, this looks the strangest (double max).

Issue Links

• relates to: FLINK-13980 FLIP-49 Unified Memory Configuration for TaskExecutors
• links to: GitHub Pull Request #13086