[KAFKA-4875] Kafka Streams: topic groups and builder.stream API

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.10.2.0
• Component/s: streams
• Assignee: Unassigned
• Reporter: Eno Thereska
• Created: 09/Mar/17 11:44
• Updated: 01/Feb/18 11:34

Description

One thing that has come up in mailing list is that the notion of topic groups is unclear. So if you have 2 topics, each with 3 partitions, you'd expect 6 tasks. However, if you do `builder.stream(topic1, topic2)` you actually get only 3 tasks created. If you do `builder.stream(topic1); builder.stream(topic2)` you get 6 tasks, i.e., parallelism is increased. So the same application, calling builder.stream() in two different ways, might see different performance.

In the Kafka Streams documentations we mention partitions and tasks, but not topic groups. We also do not document the effects of using builder.stream with a topic array. We also need to revisit whether the API and its effects are confusing.