[BEAM-8215] Wordcount 1GB Python PKB benchmarks sometimes fail with uninformative error,

• Type: Bug
• Status: Resolved
• Priority: P2
• Resolution: Won't Fix
• Fix Version/s: Not applicable
• Component/s: testing
• Labels: flake stale-assigned
• Assignee: Mark Liu
• Reporter: Valentyn Tymofieiev
• Created: 11/Sep/19 21:58
• Updated: 24/Mar/21 21:15
• Resolved: 02/Jun/20 18:25

Description

Example:
https://builds.apache.org/job/beam_PerformanceTests_WordCountIT_Py36/452/console

12:09:27 2019-09-11 19:09:27,655 a47400ce MainThread beam_integration_benchmark(1/1) ERROR    Error during benchmark beam_integration_benchmark
12:09:27 Traceback (most recent call last):
12:09:27   File "/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_WordCountIT_Py36/PerfKitBenchmarker/perfkitbenchmarker/pkb.py", line 841, in RunBenchmark
12:09:27     DoRunPhase(spec, collector, detailed_timer)
12:09:27   File "/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_WordCountIT_Py36/PerfKitBenchmarker/perfkitbenchmarker/pkb.py", line 687, in DoRunPhase
12:09:27     samples = spec.BenchmarkRun(spec)
12:09:27   File "/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_WordCountIT_Py36/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py", line 160, in Run
12:09:27     job_type=job_type)
12:09:27   File "/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_WordCountIT_Py36/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py", line 91, in SubmitJob
12:09:27     assert retcode == 0, "Integration Test Failed."
12:09:27 AssertionError: Integration Test Failed.

It seems like Job submission failed, but there are no details. I talked with markflyhigh, and sounds like we plan to stop using PKB in favor of another framework.

Assigning to Mark for now to triage follow up or reassign as appropriate.