Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-8582

Python SDK emits duplicate records for Default and AfterWatermark triggers

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: P2
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.19.0
    • Component/s: sdk-py-core
    • Labels:
      None

      Description

      This was found after fixing https://issues.apache.org/jira/browse/BEAM-8581. The fix for 8581 was to pass in the input watermark. Previously, it was using MIN_TIMESTAMP for all of its EOW calculations. By giving it a proper input watermark, this bug started to manifest.

      The DefaultTrigger and AfterWatermark do not clear their timers after the watermark passed the end of the endow, leading to duplicate records being emitted.

      Fix: Clear the watermark timer when the watermark reaches the end of the window.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                rohdesam Sam Rohde
                Reporter:
                rohdesam Sam Rohde
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: