[FLINK-18190] Document implications on derived memory values when configuring meta space

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Not A Problem
• Assignee: Unassigned
• Reporter: Chesnay Schepler
• Created: 08/Jun/20 15:10
• Updated: 09/Jun/20 10:17
• Resolved: 09/Jun/20 10:17

Description

Let's start with a base line, where we configure the total process memory to 1024mb:

Final Master Memory configuration:
  Total Process Memory:      1024.000mb (1073741824 bytes)
    Total Flink Memory:      576.000mb (603979776 bytes)
      Flink Heap Memory:     448.000mb (469762048 bytes)
      Flink Off-heap Memory: 128.000mb (134217728 bytes)
    JVM Metaspace Memory:    256.000mb (268435456 bytes)
    JVM Overhead Memory:     192.000mb (201326592 bytes)

Increasing the Metaspace Memory by 1 reduces the Total Flink memory:

Final Master Memory configuration:
  Total Process Memory:      1024.000mb (1073741824 bytes)
    Total Flink Memory:      575.000mb (602931200 bytes)
      Flink Heap Memory:     447.000mb (468713472 bytes)
      Flink Off-heap Memory: 128.000mb (134217728 bytes)
    JVM Metaspace Memory:    257.000mb (269484032 bytes)
    JVM Overhead Memory:     192.000mb (201326592 bytes)

If on the other hand we configured the Total Flink Memory as well, then the JVM Overhead is reduced instead.

This is somewhat in line with the documentation, but it may be good to explicitly document this behavior. Users that rely on the default configuration may stumble on this when they run into metaspace issues, increasing this one option without having configured anything else.

Issue Links

• is a child of: FLINK-17973 Test memory configuration of Flink cluster