Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
Kafka currently uniquely identifies a topic by its name. This is generally sufficient, but there are flaws in this scheme if a topic is deleted and recreated with the same name. As a result, Kafka attempts to prevent these classes of issues by ensuring a topic is deleted from all replicas before completing a deletion. This solution is not perfect, as it is possible for partitions to be reassigned from brokers while they are down, and there are no guarantees that this state will ever be cleaned up and will not cause issues in the future.
As the controller must wait for all replicas to delete their local partitions, deletes can also become blocked, preventing topics from being created with the same name until the deletion is complete on all replicas. This can mean that downtime for a single broker can effectively cause a complete outage for everyone producing/consuming to that topic name, as the topic cannot be recreated without manual intervention.
Unique topic IDs could help address this issue by associating a unique ID with each topic, ensuring a newly created topic with a previously used name cannot be confused with a previous topic with that name.
KIP-516: https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers