[FLINK-9761] Potential buffer leak in PartitionRequestClientHandler during job failures

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Won't Fix
• Affects Version/s: 1.5.0
• Fix Version/s: 1.6.5, 1.7.3
• Component/s: Runtime / Network
• Assignee: Unassigned
• Reporter: Nico Kruber
• Created: 05/Jul/18 10:40
• Updated: 21/Nov/19 05:39
• Resolved: 21/Nov/19 05:39

Description

PartitionRequestClientHandler#stagedMessages may be accessed from multiple threads:
1) Netty's IO thread
2) During cancellation, PartitionRequestClientHandler.BufferListenerTask#notifyBufferDestroyed is called

If PartitionRequestClientHandler.BufferListenerTask#notifyBufferDestroyed thinks, stagesMessages is empty, however, it will not install the stagedMessagesHandler that consumes and releases buffers from received messages.
Unless some unexpected combination of code calls prevents this from happening, this would leak the non-recycled buffers.