[BEAM-9445] Preoptimize causes error with environment setup

• Type: Bug
• Status: Triage Needed
• Priority: P3
• Resolution: Unresolved
• Component/s: sdk-py-core
• Assignee: Unassigned
• Reporter: Kyle Weaver
• Created: 04/Mar/20 23:22
• Updated: 13/Apr/23 10:59

Description

Setting --experiments=pre_optimize=all causes an error with StatisticsGen in the TFX taxi example pipeline [1]:

File "[redacted]/apache_beam/runners/portability/fn_api_runner_transforms.py", line 250, in executable_stage_transform
environment=components.environments[self.environment],
TypeError: None has type NoneType, but expected one of: bytes, unicode [while running 'Run[StatisticsGen]']

cc angoenka

[1] https://github.com/tensorflow/tfx/blob/ff314a6803675548c89a016a5110a91e5bf98024/tfx/examples/chicago_taxi_pipeline/taxi_pipeline_portable_beam.py#L155

Issue Links

• is related to: BEAM-7199 Better optimize Portable pipelines in Java fuser