Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-8566

Checkpoint buffer is flushed prematurely when another bundle is started during checkpointing

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: P2
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.17.0
    • Component/s: runner-flink
    • Labels:
      None

      Description

      As part of a checkpoint, the current bundle is finalized. When the bundle is finalized, the watermark, currently held back, may also be progressed which can cause the start of another bundle. When a new bundle is started, any to-be-buffered items from the previous bundle for the pending checkpoint may be emitted. This should not happen.

      This only effects portable pipelines where we have to hold back the watermark due to the asynchronous processing of elements.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                mxm Maximilian Michels
                Reporter:
                mxm Maximilian Michels
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h 40m
                  2h 40m