[FLINK-26403] SinkWriter should emit all the pending committables on endOfInput

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.15.0
• Fix Version/s: 1.15.0
• Component/s: API / Core
• Labels: pull-request-available
• Assignee: Gen Luo
• Reporter: Yun Gao
• Created: 28/Feb/22 17:34
• Updated: 04/Mar/22 09:13
• Resolved: 03/Mar/22 03:43

Description

Currently the SinkWriterOperator not drained all the pending committables on endOfInput() and left them till final checkpoint, which would be in fact deserted. This might cause data loss or the CommitterOperator hanged on endOfInput() due to not received expected number of committables.

Issue Links

• causes: FLINK-26314 StreamingCompactingFileSinkITCase.testFileSink failed on azure
• causes: FLINK-26321 KafkaSinkITCase.testWriteRecordsToKafkaWithExactlyOnceGuarantee fails on azure
• supercedes: FLINK-26321 KafkaSinkITCase.testWriteRecordsToKafkaWithExactlyOnceGuarantee fails on azure
• links to: GitHub Pull Request #18938