Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-11103 [Go SDK] Native Streaming SplittableDoFns
  3. BEAM-11104

[Go SDK] DoFn Self Checkpointing

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Triage Needed
    • P3
    • Resolution: Fixed
    • None
    • 2.40.0
    • sdk-go
    • None

    Description

      Allow SplittableDoFns to self checkpoint.

      Design doc: https://docs.google.com/document/d/1_JbzjY9JR07ZK5v7PcZevUfzHPsqwzfV7W6AouNpMPk/edit?usp=sharing

       

      Feature is written E2E and users will be able to return ProcessContinuations from SDFs as of 2.39.0 but the full behavior has not been fully validated. An integration test that validates self-checkpointing is working as-intended will need to be written and passing before the feature is no longer considered experimental and this ticket is marked as resolved.

      Attachments

        Activity

          People

            jrmccluskey Jack McCluskey
            lostluck Robert Burke
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 28h 40m
                28h 40m