[BEAM-14282] Fix exceptions swallowed in several Python I/O connectors

• Type: Bug
• Status: Resolved
• Priority: P1
• Resolution: Fixed
• Affects Version/s: 2.32.0, 2.33.0, 2.34.0, 2.35.0, 2.36.0, 2.37.0
• Fix Version/s: 2.38.0
• Component/s: io-py-gcp
• Assignee: Chamikara Madhusanka Jayalath
• Reporter: Chamikara Madhusanka Jayalath
• Created: 09/Apr/22 02:17
• Updated: 13/Apr/22 00:56
• Resolved: 13/Apr/22 00:56

Description

Seems like we do not re-throw errors after reporting metrics at following locations.

https://github.com/apache/beam/blob/8e217ea0d1f383ef5033ef507b14d01edf9c67e6/sdks/python/apache_beam/io/gcp/datastore/v1new/datastoreio.py#L303

https://github.com/apache/beam/blob/70d9e2a08cc32192790cd9c98ffa15a756877a73/sdks/python/apache_beam/io/gcp/gcsio.py#L644

Not re-raising these errors could result in data correctness issues for downstream consumers.

Issue Links

• links to: GitHub Pull Request #17329
• links to: GitHub Pull Request #17330