Details
-
Improvement
-
Status: Resolved
-
P3
-
Resolution: Fixed
-
None
-
None
Description
We have support in the Java SDK for using the BigQuery Storage API for reads, but only the target query or table is supported as a ValueProvider to be specified at runtime. AFAICT, there is no reason we can't delay specifying readOptions until runtime as well.
The readOptions are accessed by BigQueryStorageTableSource in getTargetTable; I believe that's occurring at runtime, but I'd love for someone with deeper BoundedSource knowledge to confirm that.
I'd advocate for adding new methods `TypedRead.withSelectedFields(ValueProvider<List<String>> value)` and `TypedRead.withRowRestriction(ValueProvider<String> value)`. The existing `withReadOptions` method would then populate the other two as StaticValueProviders. Perhaps we'd want to deprecate `withReadOptions` in favor or specifying individual read options as separate parameters.
Attachments
Issue Links
- links to