Minor Description
When using DSL in Kafka Streams, data re-partition happens only when key-changing operation is followed by stateful operation. On the other hand, in DSL, stateful computation can happen using transform() operation as well. Problem with this approach is that, even if any upstream operation was key-changing before calling transform(), no auto-repartition is triggered. If repartitioning is required, a call to through(String) should be performed before transform(). With the current implementation, burden of managing and creating the topic falls on user and introduces extra complexity of managing Kafka Streams application.
Attachments
Issue Links
- contains
-
KAFKA-7608 A Kafka Streams DSL transform or process call should potentially trigger a repartition
-
- Resolved
-
- fixes
-
KAFKA-6182 Automatic co-partitioning of topics via automatic intermediate topic with matching partitions
-
- Resolved
-
-
-
- Resolved
-
- is related to
-
KAFKA-10003 Deprecate KStream#through in favor of KStream#repartition
-
- Resolved
-
- links to