[FLINK-16917] "TPC-DS end-to-end test (Blink planner)" gets stuck

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 1.11.0
• Component/s: Runtime / Task, Tests
• Labels: pull-request-available test-stability
• Assignee: Arvid Heise
• Reporter: Robert Metzger
• Created: 01/Apr/20 09:48
• Updated: 22/Jun/21 14:05
• Resolved: 02/Apr/20 11:20

Description

The message you see from the CI system is

##[error]The job running on agent Hosted Agent ran longer than the maximum time of 240 minutes. For more information, see https://go.microsoft.com/fwlink/?linkid=2077134

Example: https://dev.azure.com/rmetzger/Flink/_build/results?buildId=6899&view=logs&j=c88eea3b-64a0-564d-0031-9fdcd7b8abee

The end of the log file looks as follows:

2020-03-31T23:00:40.5416207Z [INFO]Run TPC-DS query 97 success.
2020-03-31T23:00:40.5439265Z [INFO]Run TPC-DS query 98 ...
2020-03-31T23:00:40.8269500Z Job has been submitted with JobID eec4759ae6d585ee9f8d9f84f1793c0e
2020-03-31T23:01:33.4757621Z Program execution finished
2020-03-31T23:01:33.4758328Z Job with JobID eec4759ae6d585ee9f8d9f84f1793c0e has finished.
2020-03-31T23:01:33.4758880Z Job Runtime: 51093 ms
2020-03-31T23:01:33.4759057Z 
2020-03-31T23:01:33.4760999Z [INFO]Run TPC-DS query 98 success.
2020-03-31T23:01:33.4761612Z [INFO]Run TPC-DS query 99 ...
2020-03-31T23:01:33.7297686Z Job has been submitted with JobID f47efc4194df2e0ead677fff239f3dfd
2020-03-31T23:01:50.0037484Z ##[error]The operation was canceled.
2020-03-31T23:01:50.0091655Z ##[section]Finishing: Run e2e tests

Notice the time difference between "Job has been submitted" and "The operation was canceled.". There was nothing happening for 20 minutes.

Issue Links

• duplicates: FLINK-16910 The job running on agent Hosted Agent ran longer than the maximum time of 240 minutes
• relates to: FLINK-16795 End to end tests timeout on Azure
• links to: GitHub Pull Request #11611