[KAFKA-6035] Avoid creating changelog topics for state stores that are directly piped to a sink topic

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: streams
• Assignee: Jeyhun Karimov
• Reporter: Guozhang Wang
• Created: 10/Oct/17 00:34
• Updated: 06/Apr/18 22:52

Description

Today Streams make all state stores to be backed by a changelog topic by default unless users overrides it by disableLogging when creating the state store / materializing the KTable. However there are a few cases where a separate changelog topic would not be required as we can re-use an existing topic for that. This ticket summarize a specific issue that can be optimized:

Consider the case when a KTable is materialized and then sent directly into a sink topic with the same key, e.g.

table1 = stream.groupBy(...).aggregate("state1").to("topic2");

Then we do not need to create a state1-changelog but can just use topic2 as its changelog.

Issue Links

• relates to: KAFKA-6761 Reduce Kafka Streams Footprint
• links to: PR