[KAFKA-8810] Add mechanism to detect topology mismatch between streams instances

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: streams
• Assignee: Unassigned
• Reporter: Vinoth Chandar
• Created: 16/Aug/19 15:49
• Updated: 08/Feb/20 03:52

Description

Noticed this while reading through the StreamsPartitionAssignor related code. If an user accidentally deploys a different topology on one of the instances, there is no mechanism to detect this and refuse assignment/take action. Given Kafka Streams is designed as an embeddable library, I feel this is rather an important scenario to handle. For e.g, kafka streams is embedded into a web front end tier and operators deploy a hot fix for a site issue to a few instances that are leaking memory and that accidentally also deploys some topology changes with it.

Please feel free to close the issue, if its a duplicate. (Could not find a ticket for this)

Issue Links

• is blocked by: KAFKA-8307 Kafka Streams should provide some mechanism to determine topology equality and compatibility
• is related to: KAFKA-9518 NullPointerException on out-of-order topologies
• relates to: KAFKA-7669 Stream topology definition is not robust to the ordering changes