[KAFKA-13394] Topic IDs should be removed from PartitionFetchState if they are no longer sent by the controller

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 3.1.0
• Fix Version/s: 3.1.0
• Assignee: Justine Olshan
• Reporter: Justine Olshan
• Reviewer: David Jacot
• Created: 22/Oct/21 18:49
• Updated: 18/Nov/21 16:33
• Resolved: 18/Nov/21 16:33

Description

With KAFKA-13102, we added topic IDs to the InitialFetchState and the PartitionFetchState in order to send fetch requests using topic IDs when ibp is 3.1. 

However, there are some cases where we could initially send topic IDs from the controller and then no longer to do so (controller changes to an IBP < 2.8). If we do not remove from the PartitionFetchState and one broker is still IBP 3.1, it will try to send a version 13 fetch request to brokers that no longer have topic IDs in the metadata cache. This could leave the cluster in a state unable to fetch from these partitions.

Issue Links

• links to: GitHub Pull Request #11459