[FLINK-16614] FLIP-116 Unified Memory Configuration for Job Manager

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.11.0
• Component/s: Runtime / Configuration, Runtime / Coordination
• Labels: Umbrella

Release Note

see comment: https://issues.apache.org/jira/browse/FLINK-16614?focusedCommentId=17117630&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17117630

• Assignee: Andrey Zagrebin
• Reporter: Andrey Zagrebin
• Created: 16/Mar/20 10:49
• Updated: 16/Nov/20 14:18
• Resolved: 05/Nov/20 07:41

Description

This is the umbrella issue of FLIP-116: Unified Memory Configuration for Job Managers.

Issue Links

• blocks: FLINK-16734 Remove ClusterSpecification
• contains: FLINK-9904 Allow users to control MaxDirectMemorySize
• is related to: FLINK-18153 flink-daemon does not detect errors if JVM crashes immediately
• is related to: FLINK-18191 Rename new Master memory options to start with "master" prefix
• is related to: FLINK-18219 Improve exception if jar submission hits OOM
• is related to: FLINK-18250 Enrich OOM error messages with more details in ClusterEntrypoint
• is related to: FLINK-18353 [1.11.0] maybe document jobmanager behavior change regarding -XX:MaxDirectMemorySize
• is related to: FLINK-20171 Improve error message for Flink process memory configuration
• relates to: FLINK-17973 Test memory configuration of Flink cluster
• relates to: FLINK-13980 FLIP-49 Unified Memory Configuration for TaskExecutors
• relates to: FLINK-16754 Consider refactoring of ProcessMemoryUtilsTestBase to avoid inheritance

Sub-Tasks

• Introduce data structures and utilities to calculate Job Manager memory components: Closed
• Extend and use BashJavaUtils to start JM JVM process and pass JVM memory args: Closed
• Use JobManagerProcessUtils to start JM container and pass JVM memory args: Closed
• Deprecate legacy heap memory option for JM and expose the new ones in docs: Closed
• Add memory related JVM args to Mesos JM startup scripts: Closed
• Update user documentation for job manager memory model: Resolved
• Update Chinese user documentation for job manager memory model: Closed