Details
-
Bug
-
Status: Open
-
P3
-
Resolution: Unresolved
-
None
-
None
-
None
Description
Java SDK has maxStreamingBatchSize.
Implementing something similar for Python would prevent 400 HTTP errors from BigQuery when batches are too large. Instead, Beam should try to split batches and log a warning (once) when trying to insert a single very large row.