[FLINK-28980] Release Testing: Verify FLIP-168 speculative execution

• Type: Sub-task
• Status: Closed
• Priority: Blocker
• Resolution: Done
• Fix Version/s: 1.16.0
• Component/s: Runtime / Coordination
• Labels: release-testing
• Assignee: Biao Liu
• Reporter: Zhu Zhu
• Created: 16/Aug/22 03:33
• Updated: 01/Sep/22 02:58
• Resolved: 01/Sep/22 02:58

Description

Speculative execution is introduced in Flink 1.16 to deal with temporary slow tasks caused by slow nodes. This feature currently consists of 4 FLIPs:

• FLIP-168: Speculative Execution core part
• FLIP-224: Blocklist Mechanism
• FLIP-245: Source Supports Speculative Execution
• FLIP-249: Flink Web UI Enhancement for Speculative Execution

This ticket aims for verifying FLIP-168, along with FLIP-224 and FLIP-249.

More details about this feature and how to use it can be found in this documentation.

To do the verification, the process can be:

• Write a Flink job which has a subtask running much slower than others (e.g. sleep indefinitely if it runs on a certain host, the hostname can be retrieved via InetAddress.getLocalHost().getHostName(), or if its (subtaskIndex + attemptNumer) % 2 == 0)
• Modify Flink configuration file to enable speculative execution and tune the configuration as you like
• Submit the job. Checking the web UI, logs, metrics and produced result.