Details
-
Bug
-
Status: Resolved
-
P1
-
Resolution: Fixed
-
2.1.0, 2.2.0, 2.3.0, 2.4.0
-
None
Description
WatermarkHold.addElementHold and WatermarkHold.addGarbageCollectionHold take to not add holds that would be before the input watermark.
However WatermarkHold.onMerge calls StateMerging.mergeWatermarks which if the window depends only on window, sets a hold for the end of the window regardless of the input watermark.
Thus if you have a WindowingStrategy such as:
WindowingStrategy.of(Sessions.withGapDuration(gapDuration))
.withMode(AccumulationMode.DISCARDING_FIRED_PANES)
.withTrigger(
Repeatedly.forever(
AfterWatermark.pastEndOfWindow()
.withLateFirings(AfterPane.elementCountAtLeast(10))))
.withAllowedLateness(allowedLateness))
and you merge windows that are late, you might end up holding the watermark until the allowedLateness has passed.
Attachments
Issue Links
- links to