[BEAM-10999] Bigquery Warning messages cause Dataflow job to fail

• Type: Bug
• Status: Triage Needed
• Priority: P3
• Resolution: Unresolved
• Affects Version/s: 2.19.0, 2.24.0
• Component/s: io-java-gcp
• Environment: Google Dataflow
• External issue URL: https://stackoverflow.com/questions/64158343/gcp-dataflow-failing-due-to-bigquery-warning-message-about-cloud-sql-federated-c
• Assignee: Unassigned
• Reporter: Robert Coe
• Created: 01/Oct/20 15:48
• Updated: 04/Jun/22 18:07

Description

I have a job fails when BigQuery throws any warning messages, even if the query is successful.

org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers.parseStatus Checks for "getErrors" and sets status to failure if there are any. However, this can include warning messages.

[Google's Bigquery Client Code|https://github.com/googleapis/google-api-java-client-services/blob/master/clients/google-api-services-bigquery/v2/1.30.1/com/google/api/services/bigquery/model/JobStatus.java] states that 

"Errors here do not necessarily mean that the job has completed or was unsuccessful."