[FLINK-15989] Rewrap OutOfMemoryError in allocateUnpooledOffHeap with better message

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.10.1, 1.11.0
• Component/s: Runtime / Coordination
• Labels: pull-request-available
• Assignee: Andrey Zagrebin
• Reporter: Andrey Zagrebin
• Created: 11/Feb/20 08:49
• Updated: 24/Mar/20 07:54
• Resolved: 24/Mar/20 07:54

Description

Now if Flink allocates direct memory in MemorySegmentFactory#allocateUnpooledOffHeapMemory and its limit is exceeded for any reason, e.g. user code over-allocated direct memory, ByteBuffer#allocateDirect will throw a generic "OutOfMemoryError: Direct buffer memory". We can catch it and add a message which provides more explanation and points to an option taskmanager.memory.task.off-heap.size to increase as a possible solution.

Issue Links

• relates to: FLINK-13980 FLIP-49 Unified Memory Configuration for TaskExecutors
• relates to: FLINK-15981 Control the direct memory in FileChannelBoundedData.FileBufferReader
• links to: GitHub Pull Request #11408
• links to: ML report
• links to: ML response