[BEAM-3167] Update BigQueryIO to support schema auto detection when executing load jobs

• Type: Improvement
• Status: Open
• Priority: P3
• Resolution: Unresolved
• Component/s: io-java-gcp
• Assignee: Unassigned
• Reporter: Chamikara Madhusanka Jayalath
• Created: 09/Nov/17 20:14
• Updated: 03/Jun/22 19:19

Description

BigQuery supports detecting unspecified schemas automatically when executing load jobs: https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs#configuration.load

By default this is false and Dataflow does not set this to true when creating load jobs: https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/WriteTables.java#L228

Probably we should support specifying this as an option in BigQueryIO.