[BEAM-7850] Make Environment a top level attribute of PTransform

• Type: Sub-task
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.19.0
• Component/s: beam-model
• Assignee: Chamikara Madhusanka Jayalath
• Reporter: Chamikara Madhusanka Jayalath
• Created: 30/Jul/19 16:24
• Updated: 13/Apr/21 20:05
• Resolved: 24/Jan/20 22:22

Description

Currently Environment is not a top level attribute of the PTransform (of runner API proto).

https://github.com/apache/beam/blob/master/model/pipeline/src/main/proto/beam_runner_api.proto#L99

Instead it is hidden inside various payload objects. For example, for ParDo, environment will be inside SdkFunctionSpec of ParDoPayload.

https://github.com/apache/beam/blob/master/model/pipeline/src/main/proto/beam_runner_api.proto#L99

 

This makes tracking environment of different types of PTransforms harder and we have to fork code (on the type of PTransform) to extract the Environment where the PTransform should be executed. It will probably be simpler to just make Environment a top level attribute of PTransform.

Issue Links

• links to: GitHub Pull Request #10183
• links to: GitHub Pull Request #10452
• links to: GitHub Pull Request #10453