[FLINK-28942] Deadlock may occurs when releasing readers for SortMergeResultPartition

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.16.0
• Fix Version/s: 1.16.0
• Labels: pull-request-available
• Assignee: Yuxin Tan
• Reporter: Yuxin Tan
• Created: 12/Aug/22 04:26
• Updated: 12/Aug/22 09:53
• Resolved: 12/Aug/22 09:52

Description

After adding the logic of recycling buffers in CompositeBuffer in https://issues.apache.org/jira/browse/FLINK-28373, when reading data and recycling buffers simultaneously, the deadlock between the lock of SortMergeResultPartition and the lock of SingleInputGate may occur.

In short, the deadlock may occur as follows.

1. SingleInputGate.getNextBufferOrEvent (SingleInputGate lock)

CompositeBuffer.getFullBufferData -> CompositeBuffer.recycleBuffer -> waiting for 

SortMergeResultPartition lock;

2. ResultPartitionManager.releasePartition (SortMergeResultPartition lock) -> 

SortMergeSubpartitionReader.notifyDataAvailable -> 

SingleInputGate.notifyChannelNonEmpty -> waiting for SingleInputGate lock.

The possibility of this deadlock is very small, but we should fix the bug as soon as possible.

Issue Links

• is caused by: FLINK-28373 Read a full buffer of data per file IO read request for sort-shuffle
• links to: GitHub Pull Request #20560