[FLINK-10884] Flink on yarn TM container will be killed by nodemanager because of the exceeded physical memory.

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 1.5.5, 1.6.2, 1.7.0
• Component/s: Deployment / YARN, Runtime / Coordination
• Labels: pull-request-available yarn

Environment

version  : 1.6.2 

module : flink on yarn

centos  jdk1.8

hadoop 2.7

• Assignee: wgcn
• Reporter: wgcn
• Created: 14/Nov/18 14:29
• Updated: 28/May/19 07:17
• Resolved: 28/May/19 07:17

Description

TM container will be killed by nodemanager because of  the exceeded physical memory. I found the lanuch context lanuching TM container that "container memory = heap memory + offHeapSizeMB" at the class org.apache.flink.runtime.clusterframework.ContaineredTaskManagerParameters from line 160 to 166  I set a safety margin for the whole memory container using. For example if the container limit 3g memory, the sum memory that "heap memory + offHeapSizeMB"  is equal to 2.4g to prevent the container being killed. Do we have the ready-made solution or I can commit my solution.

Issue Links

• is related to: FLINK-10317 Configure Metaspace size by default
• links to: GitHub Pull Request #7185