[FLINK-13984] Separate on-heap and off-heap managed memory pools

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Done
• Fix Version/s: 1.10.0
• Component/s: Runtime / Task
• Labels: pull-request-available

Release Note

The managed memory segment preallocation has been completely removed (Configuration option 'taskmanager.memory.preallocate'). It should not break the existing setups but setting this option will have no effect which can lead to a slight performance degradation as the segments will be allocated only in the ad-hoc manner during the execution.

• Assignee: Andrey Zagrebin
• Reporter: Xintong Song
• Created: 06/Sep/19 02:54
• Updated: 22/Oct/19 12:06
• Resolved: 22/Oct/19 12:06

Description

• Update MemoryManager to have two separated pools.
• Extend MemoryManager interfaces to specify which pool to allocate memory from.

Implement this step in common code paths for the legacy / new mode. For the legacy mode, depending to the configured memory type, we can set one of the two pools to the managed memory size and always allocate from this pool, leaving the other pool empty.

Issue Links

• is related to: FLINK-14492 Modify state backend to report reserved memory to MemoryManager
• links to: GitHub Pull Request #9693
• links to: pull request #9693