Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-3301 Go SplittableDoFn support
  3. BEAM-11020

Fix multi-window SDF splitting in Go.

Attach filesAttach ScreenshotVotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: P2
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: Not applicable
    • Component/s: sdk-go
    • Labels:
      None

      Description

      When splitting an element that's being processed on multiple windows, currently the element gets split based on the current progress of the current element+window pair, and the primary and residual both contain all windows from the original. This is incorrect as certain windows in the residual have already been processed.

      The correct way to perform this split is to put the element + all fully processed windows in the primary, the element + all fully unprocessed windows in the residual, and then only split the element at the current element + window pair.

      This fix should already be in Java and Python, so those can be referenced.

        Attachments

          Activity

            People

            • Assignee:
              danoliveira Daniel Oliveira
              Reporter:
              danoliveira Daniel Oliveira

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 2h 10m
                2h 10m

                  Issue deployment