Details

    • Sub-task
    • Status: Resolved
    • P3
    • Resolution: Implemented
    • None
    • 2.39.0
    • sdk-go
    • None

    Description

      Eg. to support acking pubsub/kafka messages as processed after the results have been properly committed by the runner.

      Note, that due to BEAM-10959 that when implementing this, an instruction must remain "active" until it's finalization occurs as well. Specifically, we should probably keep another map around for "to be finalized" process bundle instructions so we can return the appropriate "empty" response and not accidently evict them from the nearly equivalent inactive state until after finalization.

      https://s.apache.org/beam-finalizing-bundles

       

      (To be updated once https://github.com/apache/beam/pull/13160 is merged and the programming guide updated with SDF content.)
      See also Java and Python approaches
      https://beam.apache.org/documentation/programming-guide/#bundle-finalization

      Attachments

        Activity

          People

            damccorm Danny McCormick
            lostluck Robert Burke
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 15.5h
                15.5h