[FLINK-2667] Rework configuration parameters

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Won't Do
• Affects Version/s: 0.10.0
• Component/s: Deployment / Scripts
• Assignee: Maximilian Michels
• Reporter: Maximilian Michels
• Created: 14/Sep/15 17:11
• Updated: 28/Feb/19 16:02
• Resolved: 28/Feb/19 16:02

Description

In the course of FLINK-2641, we came up with some changes to make the configuration parameters more meaningful and self-explanatory. Some concerns about backwards-compatibility were raised in the associated pull request: https://github.com/apache/flink/pull/1125. That's why I decided to delay those changes.

Here are the changes to the configuration which I would like to propose:

taskmanager.memory.size --> taskmanager.memory.managed

taskmanager.memory.fraction --> taskmanager.memory.managed.fraction

taskmanager.heap.mb --> taskmanager.memory
(Change its meaning to combined JVM heap + off-heap memory)

jobmanager.heap.mb --> jobmanager.memory

taskmanager.network.numberOfBuffers --> taskmanager.network.memory
(Specify the network size in terms of space and not in terms of the number of buffers)

I think those changes would make configuration easier and improve the overall user experience of Flink. The drawback is that it requires some users to update their config files. I believe the negative impact will be very little because only two of the changed parameters are present in the default config. Adapting to these new parameters should be easy because they make a lot more sense but I wanted to put this up for debate.