Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
KIP-516 introduces topic IDs to Kafka. We are in the process of updating many of the protocols to support them. In most cases, we are dropping the topic name entirely from new API versions. I think there are two open questions for KIP-500 in regard to this:
1. Can we assume topic ID existence in KIP-500?
I think the answer here is yes, and the existing code already assumes it. The nice thing is that KIP-516 brings with it the logic to create topic IDs for existing topics. We can rely on this ability in the bridge release to ensure that all topics have topic IDs. And we can add it to pre-upgrade validations.
2. What topic ID should be used for `@metadata`?
There are basically two options for this: either use a sentinel topic ID or let the controller generate a new one and write it into a `TopicRecord` when the cluster first initializes. If we assume long term that we won't be able to use topic names in the inter-broker protocol, then a sentinel might really be a necessity since brokers would need to know the topic ID before they can send fetches. In other words, if we generate a unique ID, then we probably still need some sentinel so that followers can fetch the initial `TopicRecord` which contains the ID.
Attachments
Issue Links
- links to