Minor Description
Windows are kept until their retention time passed. If a late arriving record is processed that is older than any window kept, a new window is created containing this single late arriving record, the aggregation is computed and the window is immediately discarded afterward (as it is older than retention time).
This behavior might case problems for downstream application as the original window aggregate might we overwritten with the late single-record- aggregate value. Thus, we should rather not process the late arriving record for this case.
However, data loss might not be acceptable for all use cases. In order to enable the use to not lose any data, window operators should allow to register a handler function that is called instead of just dropping the late arriving record.
Attachments
Issue Links
- Blocked
-
KAFKA-7222 KIP-328: Add Window Grace Period (and deprecate Window Retention)
-
- Resolved
-