Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
Flink currently provides only one build-in timestamp extractor, which assumes strictly ascending timestamps. In real world use cases, timestamps are almost never strictly ascending.
Therefore, I propose to provide an utility watermark extractor which is generating watermarks with a fixed-time trailing.
The implementation should keep track of the highest event-time seen so far and subtract a fixed amount of time from that event time.
This way, users can for example specify that the watermarks should always "lag behind" 10 minutes.