[FLINK-9676] Deadlock during canceling task and recycling exclusive buffer

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.5.0
• Fix Version/s: 1.5.1, 1.6.0
• Component/s: Runtime / Network
• Labels: pull-request-available
• Assignee: Nico Kruber
• Reporter: Zhijiang
• Created: 27/Jun/18 09:42
• Updated: 10/Jul/18 10:32
• Resolved: 05/Jul/18 10:24

Description

It may cause deadlock between task canceler thread and task thread.

The detail is as follows:

Task canceler thread -> IC1#releaseAllResources -> recycle floating buffers -> lock(LocalBufferPool#availableMemorySegments) -> IC2#notifyBufferAvailable > try to lock(IC2#bufferQueue)

Task thread -> IC2#recycle -> lock(IC2#bufferQueue) -> bufferQueue#addExclusiveBuffer -> floatingBuffer#recycleBuffer -> try to lock(LocalBufferPool#availableMemorySegments)

One solution is that listener#notifyBufferAvailable can be called outside the synchronized(availableMemorySegments) in LocalBufferPool#recycle.

The existing RemoteInputChannelTest#testConcurrentOnSenderBacklogAndRecycle can cover this case but the deadlock probability is very low, so this UT is not stable.

Issue Links

• links to: GitHub Pull Request #6254
• links to: GitHub Pull Request #6257