[FLINK-30823] Enable speculative execution for some of the typical built-in sinks

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Done
• Fix Version/s: 1.17.0
• Component/s: Runtime / Coordination
• Labels: pull-request-available
• Assignee: Biao Liu
• Reporter: Biao Liu
• Created: 30/Jan/23 10:00
• Updated: 31/Jan/23 11:19
• Resolved: 31/Jan/23 11:19

Description

After the Sink supports speculative execution, here we enable speculative execution for some built-in sinks. For each type of Sink (SinkV2, SinkFunction, OutputFormat) we picked some typical sinks that do not require any or only minor changes.

In this ticket, we would enable DiscardingSink, PrintSinkFunction, PrintSink, FileSink, FileSystemOutputFormat to support speculative execution.

Speculative execution might not be meaningful for some of these sinks, like DiscardingSink, PrintSinkFunction and PrintSink. However, when these sinks are chained with other operators, the chained task could not support speculative execution if there is any operator in chain does not support speculative execution. That's the main idea behind this.

FileSink and FileSystemOutputFormat are the most typical implementation for SinkV2 and OutputFormat. We would not enable speculative execution for the StreamingFileSink, because it's marked as deprecated. The HiveTableSink without compaction could also support speculative, because it depends on FileSystemOutputFormat.

Issue Links

• links to: GitHub Pull Request #21796