Running Python Dataflow load test in streaming option timeouts on Jenkins on case 2:
It iterates same ParDo step sequentially.
Jenkins jobs has 2h timeout. Second case usually is cancelled after 1h 47 min. The most suspicious metric here is throughput which in comparison to other jobs doesn't look steady. Sometimes there are spike after 1 hour of non action, or there are several spikes (to 30 000 elements/sec).
Python batch case scenario takes ~56 minutes, with steady throughput ~7000 elements/sec for almost whole job run.
In comparison Java same test case takes ~6 minutes. Here throughput goes up to ~100 000 elements/sec then after processing all elements it decreases.