[FLINK-23527] Clarify `SourceFunction#cancel()` contract about interrupting

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.13.1
• Fix Version/s: 1.14.0, 1.13.3
• Component/s: API / DataStream, Documentation
• Labels: pull-request-available

Release Note

Contract of the SourceFunction#cancel() method with respect to interruptions has been clarified:
- source itself shouldn’t be interrupting the source thread
- interrupt shouldn’t be expected in the clean cancellation case

• Assignee: Stephan Ewen
• Reporter: Piotr Nowojski
• Created: 28/Jul/21 10:06
• Updated: 14/Mar/23 12:38
• Resolved: 07/Sep/21 15:30

Description

We should clarify the contract of SourceFunction#cancel()

1. source itself shouldn’t be interrupting the source thread
2. interrupt shouldn’t be expected in the clean cancellation case

Interrupting the code on the clean shutdown path can cause failures when doing `stop-with-savepoint`. If source thread is interrupted during backpressure, this leaves network stack in invalid state, making it impossible to send EndOfPartitionEvent to complete the shutdown.

In a bit more detail, when source thread is backpressured, network stack might have already sent a partial record and it could be waiting for a buffer to finish writing/serialising that record. If network stack is interrupted while waiting for that buffer, it will never resume writing/serialisation of the remaining part of that record, while downstream node will be expecting those bytes. If in this situation we attempt to emit anything (like EndOfPartitionEvent), this will most likely cause deserialisation errors on the downstream nodes.

Issue Links

• is related to: FLINK-21028 Streaming application didn't stop properly
• is related to: FLINK-23528 stop-with-savepoint can fail with FlinkKinesisConsumer
• relates to: FLINK-28758 FlinkKafkaConsumer fails to stop with savepoint
• relates to: FLINK-24182 Tasks canceler should not immediately interrupt
• links to: GitHub Pull Request #16972