[BEAM-9638] False positives in worker region & zone tests

• Type: Bug
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.21.0
• Component/s: testing
• Assignee: Kyle Weaver
• Reporter: Kyle Weaver
• Created: 30/Mar/20 21:33
• Updated: 16/May/20 14:07
• Resolved: 08/Apr/20 20:54

Description

This is hypothetical (I haven't seen this actually happen), but worker region & zone tests in the Python SDK could yield false positives because they only check that errors occurred. They do not check which errors occurred. The pipeline options under test raise other expected errors: ['Missing required option: project.', 'Missing GCS path option: temp_location.', 'Missing GCS path option: staging_location.'] in addition to the error we're actually testing for. Therefore, if the code under test were to break, the tests would still pass.

https://github.com/apache/beam/blob/e2ff3967609df50ec55d18cc58fe8359ef4cacb8/sdks/python/apache_beam/options/pipeline_options_validator_test.py#L379

Issue Links

• links to: GitHub Pull Request #11267