[BEAM-9358] BigQueryIO potential write speed regression

• Type: Task
• Status: Resolved
• Priority: P3
• Resolution: Resolved
• Affects Version/s: 2.19.0
• Fix Version/s: 2.19.0
• Component/s: io-py-gcp
• Assignee: Unassigned
• Reporter: Kirill Kozlov
• Created: 21/Feb/20 23:58
• Updated: 24/Jul/20 19:57
• Resolved: 24/Feb/20 17:15

Description

There is a drastic decrease in Megabytes/second write speeds (from ~50 to ~5) [1], as well as 10x increase in runtime [2] for python BigQueryIO in the PerfKit dashboard.

Seems to be fairly recent, started on the Feb 20th and continued on the Feb 21st. Maybe a flake, but still worth investigating.

[1] https://apache-beam-testing.appspot.com/explore?dashboard=5667383922393088&widget=1939451611&container=847031938

[2] https://apache-beam-testing.appspot.com/explore?dashboard=5667383922393088&widget=2088160722&container=15365888