[FLINK-9636] Network buffer leaks in requesting a batch of segments during canceling

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.5.0, 1.6.0
• Fix Version/s: 1.5.1, 1.6.0
• Component/s: Runtime / Network
• Labels: pull-request-available
• Assignee: Nico Kruber
• Reporter: Zhijiang
• Created: 21/Jun/18 10:27
• Updated: 04/Jul/18 05:55
• Resolved: 04/Jul/18 05:55

Description

In NetworkBufferPool#requestMemorySegments, numTotalRequiredBuffers is increased by numRequiredBuffers first.

If InterruptedException is thrown during polling segments from the available queue, the requested segments will be recycled back to NetworkBufferPool, numTotalRequiredBuffers is decreased by the number of polled segments which is now inconsistent with numRequiredBuffers. So numTotalRequiredBuffers in NetworkBufferPool leaks in this case, and we can also decrease numRequiredBuffers to fix this bug.

 

Issue Links

• relates to: FLINK-9708 Network buffer leaks when buffer request fails during buffer redistribution
• links to: GitHub Pull Request #6238