[FLINK-23811] Handle finished subtasks in CommonTestUtils.waitForAllTaskRunning

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.14.0
• Fix Version/s: 1.14.0
• Component/s: Tests
• Labels: pull-request-available
• Assignee: Roman Khachatryan
• Reporter: Roman Khachatryan
• Created: 16/Aug/21 11:21
• Updated: 24/Aug/21 10:18
• Resolved: 23/Aug/21 12:25

Description

 CommonTestUtils.waitForAllTaskRunning returns when all the subtasks are running AND the job is running and not finished. However, with FLIP-147, subtasks may finish and the job will still be running. So the method won't return and instead timeout.

 

The solution could be:

• For new tests that can have finished subtasks treat return if subtask is RUNING || FINISHED
• For old tests (that assume no finished subtasks) throw an exception

Note that a subtask may be in some other state (e.g. CANCELLED) which is fine, as it can change after failing over the job.

 

This change is extracted from FLINK-21090 into a separate ticket because multiple IT cases might be affected.

Issue Links

• blocks: FLINK-21090 Add E2E/ITCase test for checkpoints after tasks finished
• links to: GitHub Pull Request #16845