Details
-
Improvement
-
Status: Resolved
-
P2
-
Resolution: Won't Fix
-
None
-
None
Description
ATM `BigQuerySource` in Python SDK can only be used by Dataflow runner. Because of this it's impossible to read data from BigQuery on portable runners, such as Flink.
The goal is to add a transform, that follows BQ source implementation in Java[1]: use a BigQuery export job to take a snapshot of the table on GCS, and then read from each produced file.