[BEAM-11252] Writing to BigQuery raises a TypeError "GlobalWindow -> ._IntervalWindowBase"

• Type: Bug
• Status: Open
• Priority: P3
• Resolution: Unresolved
• Affects Version/s: 2.24.0, 2.25.0, 2.26.0, 2.27.0, 2.28.0, 2.29.0
• Component/s: io-py-gcp
• Labels: bigquery
• Assignee: Unassigned
• Reporter: Tudor Plugaru
• Created: 12/Nov/20 10:13
• Updated: 04/Jun/22 18:21

Description

I have a pipeline that reads from a PubSub topic, does some transforms and then writes to BigQuery. The exception message is this: Cannot convert GlobalWindow to apache_beam.utils.windowed_value._IntervalWindowBase

You can find the main part of the pipeline here https://gist.github.com/PlugaruT/4666406bd8792b7b196dc1519c8885a2

The first part is basically reading from Pub/Sub and then I apply this PTranform.

And here is the gigantic stack trace: https://gist.github.com/PlugaruT/52bf3834eec95fd5cc5779d3332e1433

 

I've tried to downgrade to 2.24.0 version but this is still happening.

Also, I can't reproduce the exception locally when I run the pipeline with DirectRunner. It's happening only when running on Dataflow.