[BEAM-3376] Better expose BigQuery quota errors to users

• Type: Bug
• Status: Open
• Priority: P3
• Resolution: Unresolved
• Component/s: io-java-gcp
• Assignee: Unassigned
• Reporter: Chamikara Madhusanka Jayalath
• Created: 19/Dec/17 23:58
• Updated: 14/Feb/22 12:45

Description

Currently users simply see following error.

(9e02ab71e036fcc3): Workflow failed. Causes: (9e02ab71e036fbad): S27:Save to BigQuery/BatchLoads/MultiPartitionsReshuffle/GroupByKey/Read+Save to BigQuery/BatchLoads/MultiPartitionsReshuffle/GroupByKey/GroupByWindow+Save to BigQuery/BatchLoads/MultiPartitionsReshuffle/ExpandIterable+Save to BigQuery/BatchLoads/MultiPartitionsWriteTables/ParMultiDo(WriteTables)+Save to BigQuery/BatchLoads/MultiPartitionsWriteTables/WithKeys/AddKeys/Map+Save to BigQuery/BatchLoads/MultiPartitionsWriteTables/Window.Into()/Window.Assign+Save to BigQuery/BatchLoads/MultiPartitionsWriteTables/GroupByKey/Reify+Save to BigQuery/BatchLoads/MultiPartitionsWriteTables/GroupByKey/Write+Save to BigQuery/BatchLoads/ReifyRenameInput/View.AsIterable/View.CreatePCollectionView/ParDo(ToIsmRecordForGlobalWindow) failed., (f88fcdac78f4b497): A work item was attempted 4 times without success. Each time the worker eventually lost contact with the service. The work item was attempted on: ...

(there might be additional info in worker logs).

We should update the BigQuery connector to raise an exception with a proper error message.

Issue Links

• links to: GitHub Pull Request #13081