Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-3304

Go triggering support

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: P3
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: sdk-go
    • Labels:
      None

      Description

      `Add support for triggers.

      https://beam.apache.org/documentation/programming-guide/#triggers 

      Triggers are special runner side behavior indicating how to handle data WRT the watermark and window. Commonly configuring the processing for “late data” and similar.

      These are not currently implemented for user use in the Go SDK. Reshuffle configures triggers, but it’s not accessible. A correct trigger implementation can at least re-implement Reshuffle in a user pipeline, rather than handled specially within the framework.

       

      Similar to windowing,  many of the same places as https://issues.apache.org/jira/browse/BEAM-11100 need to be modified.

      At simplest though, it's mostly a runner side construction, with less concern on the exec side, and generally much simpler. 

      Appropriate integration tests against portable runners must be implemented:

      https://github.com/apache/beam/tree/master/sdks/go/test/integration/primitives 

      And optionally add support for the configurable triggers to the the Go Direct Runner. However, the results must be compared and validated against a semantically correct runner like the python portable runner first.

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                herohde Henning Rohde
              • Votes:
                2 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated: