[FLINK-6217] ContaineredTaskManagerParameters sets off heap memory size incorrectly

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.3.0
• Component/s: Deployment / YARN
• Assignee: Haohui Mai
• Reporter: Haohui Mai
• Created: 29/Mar/17 23:21
• Updated: 05/May/17 11:49
• Resolved: 05/May/17 11:49

Description

Thanks bill.liu8904 for triaging the issue.

When taskmanager.memory.off-heap is disabled, we observed that the total memory that Flink allocates exceed the total memory of the container:

For a 8G container the JobManager starts the container with the following parameter:

$JAVA_HOME/bin/java -Xms6072m -Xmx6072m -XX:MaxDirectMemorySize=6072m ...

The total amount of heap memory plus the off-heap memory exceeds the total amount of memory of the container. As a result YARN occasionally kills the container.

Issue Links

• links to: GitHub Pull Request #3648