[FLINK-21103] E2e tests time out on azure

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Duplicate
• Affects Version/s: 1.11.3, 1.12.1, 1.13.0
• Fix Version/s: 1.13.0
• Component/s: Build System / Azure Pipelines, Tests
• Labels: pull-request-available test-stability
• Assignee: Dawid Wysakowicz
• Reporter: Dawid Wysakowicz
• Created: 22/Jan/21 16:44
• Updated: 28/Aug/21 11:11
• Resolved: 31/Mar/21 14:02

Description

https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=12377&view=logs&j=c88eea3b-64a0-564d-0031-9fdcd7b8abee&t=ff888d9b-cd34-53cc-d90f-3e446d355529

Creating worker2 ... done
Jan 22 13:16:17 Waiting for hadoop cluster to come up. We have been trying for 0 seconds, retrying ...
Jan 22 13:16:22 Waiting for hadoop cluster to come up. We have been trying for 5 seconds, retrying ...
Jan 22 13:16:27 Waiting for hadoop cluster to come up. We have been trying for 10 seconds, retrying ...
Jan 22 13:16:32 Waiting for hadoop cluster to come up. We have been trying for 15 seconds, retrying ...
Jan 22 13:16:37 Waiting for hadoop cluster to come up. We have been trying for 20 seconds, retrying ...
Jan 22 13:16:43 Waiting for hadoop cluster to come up. We have been trying for 26 seconds, retrying ...
Jan 22 13:16:48 Waiting for hadoop cluster to come up. We have been trying for 31 seconds, retrying ...
Jan 22 13:16:53 Waiting for hadoop cluster to come up. We have been trying for 36 seconds, retrying ...
Jan 22 13:16:58 Waiting for hadoop cluster to come up. We have been trying for 41 seconds, retrying ...
Jan 22 13:17:03 Waiting for hadoop cluster to come up. We have been trying for 46 seconds, retrying ...
Jan 22 13:17:08 We only have 0 NodeManagers up. We have been trying for 0 seconds, retrying ...
21/01/22 13:17:10 INFO client.RMProxy: Connecting to ResourceManager at master.docker-hadoop-cluster-network/172.19.0.3:8032
21/01/22 13:17:11 INFO client.AHSProxy: Connecting to Application History server at master.docker-hadoop-cluster-network/172.19.0.3:10200
Jan 22 13:17:11 We now have 2 NodeManagers up.
==========================================================================================
=== WARNING: This E2E Run took already 80% of the allocated time budget of 250 minutes ===
==========================================================================================
======================================================================================================
=== WARNING: This E2E Run will time out in the next few minutes. Starting to upload the log output ===
======================================================================================================
##[error]The task has timed out.
Async Command Start: Upload Artifact
Uploading 1 files
File upload succeed.
Upload '/tmp/_e2e_watchdog.output.0' to file container: '#/11824779/e2e-timeout-logs'
Associated artifact 140921 with build 12377
Async Command End: Upload Artifact
Async Command Start: Upload Artifact
Uploading 1 files
File upload succeed.
Upload '/tmp/_e2e_watchdog.output.1' to file container: '#/11824779/e2e-timeout-logs'
Associated artifact 140921 with build 12377
Async Command End: Upload Artifact
Async Command Start: Upload Artifact
Uploading 1 files
File upload succeed.
Upload '/tmp/_e2e_watchdog.output.2' to file container: '#/11824779/e2e-timeout-logs'
Associated artifact 140921 with build 12377
Async Command End: Upload Artifact
Finishing: Run e2e tests

Issue Links

• duplicates: FLINK-21316 E2E tests exceed 250 minutes.
• is related to: FLINK-21545 Running Kerberized YARN per-job on Docker test stalls on azure
• is related to: FLINK-21556 StreamingKafkaITCase hangs on azure
• is related to: FLINK-21561 Introduce cache for binaries downloaded by bash tests
• relates to: FLINK-21519 SQLClientHBaseITCase hangs on azure
• links to: GitHub Pull Request #15448