[FLINK-10317] Configure Metaspace size by default

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.5.3, 1.6.0, 1.7.0
• Fix Version/s: 1.10.0
• Component/s: Deployment / Scripts
• Assignee: Unassigned
• Reporter: Stephan Ewen
• Created: 11/Sep/18 08:28
• Updated: 20/Feb/20 14:26
• Resolved: 20/Feb/20 14:26

Description

We should set the size of the JVM Metaspace to a sane default, like -XX:MaxMetaspaceSize=256m.

If not set, the JVM offheap memory will grow indefinitely with repeated classloading and Jitting, eventually exceeding allowed memory on docker/yarn or similar setups.

It is hard to come up with a good default, however, I believe the error messages one gets when metaspace is too small are easy to understand (and easy to take action), while it is very hard to figure out why the memory footprint keeps growing steadily and infinitely.

Issue Links

• is related to: FLINK-9080 Flink Scheduler goes OOM, suspecting a memory leak
• relates to: FLINK-11205 Task Manager Metaspace Memory Leak
• relates to: FLINK-10884 Flink on yarn TM container will be killed by nodemanager because of the exceeded physical memory.