Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.0.0
    • Fix Version/s: 0.10.0.0
    • Component/s: streams
    • Labels:
      None

      Description

      KTable is a special type of the stream that represents a changelog of a database table (or a key-value store).

      A changelog has to meet the following requirements.

      • Key-value mapping is surjective in the database table (the key must be the primary key).
      • All insert/update/delete events are delivered in order for the same key
      • An update event has the whole data (not just delta).
      • A delete event is represented by the null value.

      KTable does not necessarily materialized as a local store. It may be materialized when necessary. (see below)

      KTable supports look-up by key. KTable is materialized implicitly when look-up is necessary.

      • KTable may be created from a topic. (Base KTable)
      • KTable may be created from another KTable by filter(), filterOut(), mapValues(). (Derived KTable)
      • A call to the user supplied function is skipped when the value is null since such an event represents a deletion.
      • Instead of dropping, events filtered out by filter() or filterOut() are converted to delete events. (Can we avoid this?)
      • map(), flatMap() and flatMapValues() are not supported since they may violate the changelog requirements

      A derived KTable may be persisted to a topic by to() or through(). through() creates another base KTable.

      KTable can be converted to KStream by the toStream() method.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                yasuhiro.matsuda Yasuhiro Matsuda
                Reporter:
                yasuhiro.matsuda Yasuhiro Matsuda
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: