Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-4587

Rethink Unification of Caching with Dedupping

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: streams
    • Labels:
      None

      Description

      This is discussed in PR https://github.com/apache/kafka/pull/1588

      In order to support user-customizable state store suppliers in the DSL we did the following:

      1) Introduce a TupleForwarder to forward tuples from cached stores that is wrapping user customized stores.
      2) Narrow the scope to only dedup on forwarding if it is the default CachingXXStore with wrapper RocksDB.

      With this, the unification of dedupping and caching is less useful now, and we are complicating the inner implementations for forwarding a lot. We need to re-consider this feature with finer granularity of turning on / off caching per store, potentially with explicit triggers.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                guozhang Guozhang Wang
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated: