Description
KTable is a special type of the stream that represents a changelog of a database table (or a key-value store).
A changelog has to meet the following requirements.
- Key-value mapping is surjective in the database table (the key must be the primary key).
- All insert/update/delete events are delivered in order for the same key
- An update event has the whole data (not just delta).
- A delete event is represented by the null value.
KTable does not necessarily materialized as a local store. It may be materialized when necessary. (see below)
KTable supports look-up by key. KTable is materialized implicitly when look-up is necessary.
- KTable may be created from a topic. (Base KTable)
- KTable may be created from another KTable by filter(), filterOut(), mapValues(). (Derived KTable)
- A call to the user supplied function is skipped when the value is null since such an event represents a deletion.
- Instead of dropping, events filtered out by filter() or filterOut() are converted to delete events. (Can we avoid this?)
- map(), flatMap() and flatMapValues() are not supported since they may violate the changelog requirements
A derived KTable may be persisted to a topic by to() or through(). through() creates another base KTable.
KTable can be converted to KStream by the toStream() method.
Attachments
Issue Links
- relates to
-
KAFKA-2653 Stateful operations in the KStream DSL layer
-
- Resolved
-