[BEAM-9438] warn or error when constructor runner != options runner

• Type: Improvement
• Status: Open
• Priority: P3
• Resolution: Unresolved
• Component/s: sdk-py-core
• Assignee: Unassigned
• Reporter: Kyle Weaver
• Created: 04/Mar/20 17:19
• Updated: 04/Jun/22 14:38

Description

Right now, the runner set in the Pipeline constructor silently overrides the runner set in pipeline options [1]. This can result in subtle errors, where a different runner is used than expected [2]. I can't imagine any legitimate reason anyone would want to do this, so we should error when they contradict each other, or at least warn.

[1] https://github.com/apache/beam/blob/e20b0c213034cd438d48f4a2cb56cc902dae023f/sdks/python/apache_beam/pipeline.py#L179-L186
[2] https://github.com/tensorflow/tfx/blob/02ead9886726cf2cebd9a3d1e1b4bbd0f2cf9d00/tfx/components/base/base_executor.py#L142-L143