Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-7736

Retried work is messed up the SDKHarness work assignment

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: P2
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.14.0
    • Component/s: runner-dataflow
    • Labels:
      None

      Description

      The retried work is silently skipped in https://github.com/apache/beam/blob/f7cbf88f550c8918b99a13af4182d6efa07cd2b5/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java#L2010 

      This result into skipping of worker release work the corresponding duplicate work item and causes result into incorrect worker allocation for new work items. 

       

      This does not impact the correctness of the job but it can have an impact on a long running streaming job with multiple SDKHarness like in case of python streaming.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                angoenka Ankur Goenka
                Reporter:
                angoenka Ankur Goenka
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m