I am trying to run a pipeline (defined with the Python SDK) on Dataflow that uses beam.io.ReadFromMongoDB. When dealing with very small datasets (<10mb) it runs fine, when trying to run it with slightly larger datasets (70mb), I always get this error:
Stack trace see below. Running it on a local machine works just fine. I would highly appreciate any pointers what this could be.
I hope this is the right channel do address this.