[BEAM-11793] Batch stateful DoFn does not work with runner v2 portable job submission

• Type: Bug
• Status: Open
• Priority: P3
• Resolution: Unresolved
• Component/s: runner-dataflow
• Assignee: Unassigned
• Reporter: Yichi Zhang
• Created: 10/Feb/21 19:29
• Updated: 04/Jun/22 19:06

Description

When using Stateful DoFn in batch pipeline with dataflow runner v2, workflow will fail with

Workflow failed. Causes: S08:Reshuffle/ReshufflePerKey/GroupByKey/Read+Reshuffle/ReshufflePerKey/GroupByKey/GroupByWindow+Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps)+Reshuffle/RemoveRandomKeys+Do/Reshard/Write failed., The job failed because a work item has failed 4 times. Look in previous log entries for the cause of each one of the 4 failures. For more information, see https://cloud.google.com/dataflow/docs/guides/common-errors. The work item was attempted on these workers: 
  df2-combinefnlifecycletes-02101100-mv27-local-0
      Root cause: Shard number -598290054 is not in range [0, 20).,
  df2-combinefnlifecycletes-02101100-mv27-local-0
      Root cause: Shard number -598290054 is not in range [0, 20).,
  df2-combinefnlifecycletes-02101100-mv27-local-0
      Root cause: Shard number -598290054 is not in range [0, 20).,
  df2-combinefnlifecycletes-02101100-mv27-local-0
      Root cause: Shard number -598290054 is not in range [0, 20).