Description
Event time is assigned in stream sources via TimestampExtractor. Once the event time has been assigned, it remains the same, regardless of any downstream processing in the topology. This is insufficient for many processing jobs, particularly when the output of the job is written back into a Kafka topic, where the record's time is encoded outside of the record's value.
For instance:
- When performing windowed aggregations it may be desirable for the timestamp of the emitted record to be lower or higher limits of the time window, rather than the timestamp of the last processed element, which may be anywhere within the time window.
- When joining two streams, it is non-deterministic which of the two record's timestamps will be the timestamp of the emitted record. It would be either one depending on what order the records are processed. Even where this deterministic, it may be desirable for the emitted timestamp to be altogether different from the timestamp of the joined records. For instance, setting the timestamp to the current processing time may be desirable.
- In general, lower level processors may wish to set the timestamp of emitted records to an arbitrary value.
Attachments
Issue Links
- is duplicated by
-
KAFKA-6454 Allow timestamp manipulation in Processor API
-
- Resolved
-