[BEAM-5381] Dataflow runner creates duplicate CoGBK step IDs

• Type: Improvement
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: Not applicable
• Component/s: sdk-go
• Assignee: Robert Burke
• Reporter: Cody Schroeder
• Created: 13/Sep/18 19:51
• Updated: 16/May/20 13:43
• Resolved: 08/Apr/19 22:50

Description

https://gist.github.com/schroederc/699f42e407702cf9584b15d6885ad297

If the attached beam_dataflow_err.go pipeline is executed with the dataflow runner, GCP reports the following error:

Step with name e5 already exists. Duplicates are not allowed.

Executing the pipeline in --dry_run mode shows that "e5" is indeed duplicated. If the CoGBK in the pipeline is not scoped, the duplication is fixed.

Issue Links

• links to: GitHub Pull Request #7889
• links to: GitHub Pull Request #8238