Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-18353

[1.11.0] maybe document jobmanager behavior change regarding -XX:MaxDirectMemorySize

    XMLWordPrintableJSON

    Details

      Description

      I know FLIP-116 (Unified Memory Configuration for Job Managers) is introduced in 1.11. That does cause a small behavior change regarding `-XX:MaxDirectMemorySize`. Previously, jobmanager don't set JVM arg `-XX:MaxDirectMemorySize`, which means JVM can use up to -`Xmx` size for direct memory. Now `-XX:MaxDirectMemorySize` is always set to jobmanager.memory.off-heap.size config (default 128 mb).

       

      It is possible for jobmanager to get "java.lang.OufOfMemoryError: Direct Buffer Memory" without tuning jobmanager.memory.off-heap.size especially for high-parallelism jobs. Previously, no tuning needed.

       

      Maybe we should point out the behavior change in the migration guide?

      https://ci.apache.org/projects/flink/flink-docs-master/ops/memory/mem_migration.html#migrate-job-manager-memory-configuration

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                azagrebin Andrey Zagrebin
                Reporter:
                stevenz3wu Steven Zhen Wu
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: