[FLINK-17440] Potential Buffer leak in output unspilling for unaligned checkpoints

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.11.0
• Component/s: Runtime / Network
• Labels: pull-request-available
• Assignee: Zhijiang
• Reporter: Piotr Nowojski
• Created: 28/Apr/20 14:39
• Updated: 29/Apr/20 11:17
• Resolved: 29/Apr/20 11:17

Description

@Override
public void PipelinedSubpartition#initializeState(ChannelStateReader stateReader) throws IOException, InterruptedException {
	for (ReadResult readResult = ReadResult.HAS_MORE_DATA; readResult == ReadResult.HAS_MORE_DATA;) {
		BufferBuilder bufferBuilder = parent.getBufferPool().requestBufferBuilderBlocking();
		BufferConsumer bufferConsumer = bufferBuilder.createBufferConsumer();
		readResult = stateReader.readOutputData(subpartitionInfo, bufferBuilder); // <<<<<<<<<<<<<<<<<<<<<<<
		// check whether there are some states data filled in this time
		if (bufferConsumer.isDataAvailable()) {
			add(bufferConsumer, false, false);
			bufferBuilder.finish();
		} else {
			bufferConsumer.close();
		}
	}
}

There is a memory leak in output unspilling in case of an exception coming from the stateReader.

Issue Links

• is caused by: FLINK-16537 Implement ResultPartition state recovery for unaligned checkpoint
• links to: GitHub Pull Request #11940