[FLINK-18353] [1.11.0] maybe document jobmanager behavior change regarding -XX:MaxDirectMemorySize

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.11.0
• Fix Version/s: 1.11.0
• Component/s: Documentation, Runtime / Configuration
• Labels: pull-request-available
• Assignee: Andrey Zagrebin
• Reporter: Steven Zhen Wu
• Created: 18/Jun/20 01:11
• Updated: 24/Jun/20 08:44
• Resolved: 23/Jun/20 07:11

Description

I know FLIP-116 (Unified Memory Configuration for Job Managers) is introduced in 1.11. That does cause a small behavior change regarding `-XX:MaxDirectMemorySize`. Previously, jobmanager don't set JVM arg `-XX:MaxDirectMemorySize`, which means JVM can use up to -`Xmx` size for direct memory. Now `-XX:MaxDirectMemorySize` is always set to jobmanager.memory.off-heap.size config (default 128 mb).

 

It is possible for jobmanager to get "java.lang.OufOfMemoryError: Direct Buffer Memory" without tuning jobmanager.memory.off-heap.size especially for high-parallelism jobs. Previously, no tuning needed.

 

Maybe we should point out the behavior change in the migration guide?

https://ci.apache.org/projects/flink/flink-docs-master/ops/memory/mem_migration.html#migrate-job-manager-memory-configuration

Issue Links

• relates to: FLINK-16614 FLIP-116 Unified Memory Configuration for Job Manager
• links to: GitHub Pull Request #12745