[BEAM-11143] AfterWatermark can leak state for the late trigger when merging windows

• Type: Bug
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.26.0
• Component/s: runner-core
• Assignee: Sam Whittle
• Reporter: Sam Whittle
• Created: 27/Oct/20 20:05
• Updated: 08/Dec/20 09:34
• Resolved: 08/Dec/20 09:34

Description

AfterWatermarkStateMachine attempts to clear the late trigger state when merging and the early trigger is not closed in some windows. However it appears to clear the state corresponding to the new window, not all of the states for the input windows. This leaks the state which can lead to unbounded pipeline state buildup.

https://github.com/apache/beam/blob/68d6c8e6243b1d8f392840273f886276e2a8baff/runners/core-java/src/main/java/org/apache/beam/runners/core/triggers/AfterWatermarkStateMachine.java#L133

Issue Links

• links to: GitHub Pull Request #13206