[BEAM-8004] Flink Load tests are flaky

• Type: Bug
• Status: Resolved
• Priority: P1
• Resolution: Fixed
• Fix Version/s: 2.16.0
• Component/s: testing
• Labels: flake
• Assignee: Kamil Wasilewski
• Reporter: Lukasz Gajowy
• Created: 19/Aug/19 15:51
• Updated: 22/Aug/20 15:12
• Resolved: 20/Aug/19 12:02

Description

https://builds.apache.org/view/A-D/view/Beam/view/All/job/beam_LoadTests_Python_Combine_Flink_Batch/
https://builds.apache.org/view/A-D/view/Beam/view/All/job/beam_LoadTests_Python_GBK_Flink_Batch/

The tests are mostly failing (they sometimes succeed) due to issues with Dataproc cluster. The error: 

 root: DEBUG: java.net.UnknownHostException: beam-loadtests-python-gbk-flink-batch-68-w-10.c.apache-beam-testing.internal
13:35:25 	at java.net.InetAddress.getAllByName0(InetAddress.java:1281)
13:35:25 	at java.net.InetAddress.getAllByName(InetAddress.java:1193)
13:35:25 	at java.net.InetAddress.getAllByName(InetAddress.java:1127)
13:35:25 	at java.net.InetAddress.getByName(InetAddress.java:1077)
13:35:25 	at org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils.getRpcUrl(AkkaRpcServiceUtils.java:167)
13:35:25 	at org.apache.flink.runtime.rpc.akka.AkkaRpcServiceUtils.getRpcUrl(AkkaRpcServiceUtils.java:133)
13:35:25 	at org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.createHighAvailabilityServices(HighAvailabilityServicesUtils.java:89)
13:35:25 	at org.apache.flink.client.program.ClusterClient.<init>(ClusterClient.java:159)
13:35:25 	at org.apache.flink.client.program.rest.RestClusterClient.<init>(RestClusterClient.java:185)
13:35:25 	at org.apache.flink.client.program.rest.RestClusterClient.<init>(RestClusterClient.java:158)
13:35:25 	at org.apache.flink.client.RemoteExecutor.start(RemoteExecutor.java:152)
13:35:25 	at org.apache.flink.client.RemoteExecutor.executePlanWithJars(RemoteExecutor.java:202)
13:35:25 	at org.apache.flink.client.RemoteExecutor.executePlan(RemoteExecutor.java:187)
13:35:25 	at org.apache.flink.api.java.RemoteEnvironment.execute(RemoteEnvironment.java:173)
13:35:25 	at org.apache.beam.runners.flink.FlinkBatchPortablePipelineTranslator$BatchTranslationContext.execute(FlinkBatchPortablePipelineTranslator.java:200)
13:35:25 	at org.apache.beam.runners.flink.FlinkPipelineRunner.runPipelineWithTranslator(FlinkPipelineRunner.java:92)
13:35:25 	at org.apache.beam.runners.flink.FlinkPipelineRunner.run(FlinkPipelineRunner.java:68)
13:35:25 	at org.apache.beam.runners.fnexecution.jobsubmission.JobInvocation.runPipeline(JobInvocation.java:78)
13:35:25 	at  

Issue Links

• links to: GitHub Pull Request #9381