[BEAM-2843] Log the reason for failure for BigQuery jobs

• Type: Improvement
• Status: Open
• Priority: P3
• Resolution: Unresolved
• Component/s: io-java-gcp
• Labels: newbie starter
• Assignee: Unassigned
• Reporter: Chamikara Madhusanka Jayalath
• Created: 06/Sep/17 00:57
• Updated: 03/Jun/22 18:33

Description

For BigQuery load jobs we currently retry few times and log the job ID.

It might be good to get the reason for the failure from BigQuery (prob. this is available through status poll) and log that as well. For example, failed load jobs are logged at following location.
https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/WriteTables.java#L202