[FLINK-21793] Native K8s does not respect TM memory setups decided by RM

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.13.0
• Fix Version/s: 1.13.0
• Component/s: Deployment / Kubernetes
• Labels: pull-request-available
• Assignee: Xintong Song
• Reporter: Xintong Song
• Created: 15/Mar/21 11:34
• Updated: 16/Mar/21 12:12
• Resolved: 16/Mar/21 12:12

Description

Currently, kubernetes-taskmanager.sh decides JVM options (Xmx, Xms, etc.) from a cluster-wised configuration overwritten by TM-specific dynamic properties. The problem is that, if the cluster-wised configuration contains some unwanted options, there's no way to remove/unset them via dynamic properties, which leads to inconsistent JVM options.

One of the key improvement of FLIP-49 was to make all memory calculations happen in one place, which is the RM side for active RM modes. According to this principle, I'd suggest to generate the memory related JVM options on RM side, and passing them to TM through ENVs.

This only affects fine-grained resource management.

Issue Links

• links to: GitHub Pull Request #15214