[BEAM-6584] Python SDK creates job graphs with duplicated states when using fn_api execution mode.

• Type: Bug
• Status: Open
• Priority: P3
• Resolution: Unresolved
• Component/s: sdk-py-harness
• Assignee: Unassigned
• Reporter: Valentyn Tymofieiev
• Created: 05/Feb/19 00:39
• Updated: 03/Jun/22 23:04

Description

We observed this on apache_beam.examples.wordcount with Dataflow runner.

The graph for this wordcount job contains two steps with the same name "write/Write/WriteImpl/FinalizeWrite/MapToVoidKey1".

...
 {
        "kind": "PAR_DO_KIND",
        "id": "s41",
        "name": "write/Write/WriteImpl/FinalizeWrite/MapToVoidKey1",
        "displayData": [
          {
            "key": "fn",
            "namespace": "apache_beam.transforms.core.ParDo",
            "strValue": "apache_beam.transforms.core.CallableWrapperDoFn",
            "shortStrValue": "CallableWrapperDoFn",
            "label": "Transform Function"
          },
          {
            "key": "fn",
            "namespace": "apache_beam.transforms.core.CallableWrapperDoFn",
            "strValue": "\u003clambda\u003e",
            "label": "Transform Function"
          }
        ],
        "outputCollectionName": [
          "write/Write/WriteImpl/FinalizeWrite/MapToVoidKey1.out0"
        ],
        "inputCollectionName": [
          "write/Write/WriteImpl/Extract.out0"
        ]
      },
...
{
        "kind": "PAR_DO_KIND",
        "id": "s31",
        "name": "write/Write/WriteImpl/FinalizeWrite/MapToVoidKey1",
        "displayData": [
          {
            "key": "fn",
            "namespace": "apache_beam.transforms.core.ParDo",
            "strValue": "apache_beam.transforms.core.CallableWrapperDoFn",
            "shortStrValue": "CallableWrapperDoFn",
            "label": "Transform Function"
          },
          {
            "key": "fn",
            "namespace": "apache_beam.transforms.core.CallableWrapperDoFn",
            "strValue": "\u003clambda\u003e",
            "label": "Transform Function"
          }
        ],
        "outputCollectionName": [
          "write/Write/WriteImpl/FinalizeWrite/MapToVoidKey1.out0"
        ],
        "inputCollectionName": [
          "write/Write/WriteImpl/Extract.out0"
        ]
      },
...

CC: foegler altay robertwb