[FLINK-14631] Account for netty direct allocations in direct memory limit (Netty Shuffle)

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Won't Do
• Fix Version/s: 1.10.0
• Component/s: Runtime / Task
• Labels: pull-request-available
• Assignee: Andrey Zagrebin
• Reporter: Andrey Zagrebin
• Created: 06/Nov/19 11:08
• Updated: 27/Nov/19 10:44
• Resolved: 27/Nov/19 10:44

Description

At the moment after FLINK-13982, when we calculate JVM direct memory limit, we only account for memory segment network buffers but not for direct allocations from netty arenas in org.apache.flink.runtime.io.network.netty.NettyBufferPool. We should include netty arenas into shuffle memory calculations.

Issue Links

• links to: GitHub Pull Request #10180