[KAFKA-10721] Rewrite topology to allow for overlapping unequal topic subscriptions

• Type: Improvement
• Status: In Progress
• Priority: Minor
• Resolution: Unresolved
• Component/s: streams
• Labels: newbie++
• Assignee: Salavat Zainullin
• Reporter: A. Sophie Blee-Goldman
• Created: 13/Nov/20 19:41
• Updated: 06/Apr/21 12:18

Description

Minor followup improvement to KAFKA-6687 in which we rewrite the topology to make it possible for a user to subscribe multiple KStream/KTables to the same topic or identical set of topics. We could further extend this to make it possible for multiple KStream/KTables to be subscribed to overlapping but not identical sets of topics, ie

KStream streamA = builder.stream("topic");
KStream streamB = builder.stream("topic", "other-topic"); 

One way to do this would be to break up multiple-topic source nodes into multiple single-topic sources that get merged together in the child node.

See https://github.com/apache/kafka/pull/9582/files#diff-ac1bf2b23b80784dec20b00fdc42f2df7e5a5133d6c68978fa44aea11e950c3aR347-R349

Issue Links

• is related to: KAFKA-6687 Allow to read a topic multiple times