Description
At the moment the timestamp extractor is configured via a StreamConfig value to KafkaStreams. That means you can only have a single timestamp extractor per app, even though you may be joining multiple streams/tables that require different timestamp extraction methods.
You should be able to specify a timestamp extractor via KStreamBuilder.stream()/table(), just like you can specify key and value serdes that override the StreamConfig defaults.
KIP: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=68714788
Specifying a per-stream extractor should only be possible for sources, but not for intermediate topics. For PAPI we cannot enforce this, but for DSL through() should not allow to set a custom extractor by the user. In contrast, with regard to KAFKA-4785, is must internally set an extractor that returns the record's metadata timestamp in order to overwrite the global extractor from StreamsConfig (ie, set FailOnInvalidTimestampExtractor). This change should be done in KAFKA-4785 though.
Attachments
Issue Links
- is related to
-
KAFKA-4114 Allow for different "auto.offset.reset" strategies for different input streams
-
- Resolved
-
-
KAFKA-4785 Records from internal repartitioning topics should always use RecordMetadataTimestampExtractor
-
- Resolved
-
- links to
- mentioned in
-
Page Loading...