[FLINK-18646] Managed memory released check can block RPC thread

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.10.1, 1.11.0, 1.12.0
• Fix Version/s: 1.10.2, 1.11.2, 1.12.0
• Component/s: Runtime / Task
• Labels: pull-request-available
• Assignee: Andrey Zagrebin
• Reporter: Andrey Zagrebin
• Created: 20/Jul/20 13:20
• Updated: 26/Aug/20 15:32
• Resolved: 13/Aug/20 08:19

Description

UnsafeMemoryBudget#verifyEmpty, called on slot freeing, needs time to wait on GC of all allocated/released managed memory. If there are a lot of segments to GC then it can take time to finish the check. If slot freeing happens in RPC thread, the GC waiting can block it and TM risks to miss its heartbeat.

Issue Links

• causes: FLINK-19055 MemoryManagerSharedResourcesTest contains three tests running extraordinary long
• is caused by: FLINK-15758 Investigate potential out-of-memory problems due to managed unsafe memory allocation
• links to: GitHub Pull Request #12980