[KAFKA-17094] Make it possible to list registered KRaft nodes in order to know which nodes should be unregistered

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.7.1
• Fix Version/s: 4.0.0
• Assignee: Rikhi Ram Satnami
• Reporter: Jakub Scholz
• Created: 05/Jul/24 18:56
• Updated: 09/Aug/24 21:48

Description

Kafka seems to require nodes that are removed from the cluster to be unregistered using the Kafka Admin API. If they are unregistred, that you might run into problems later. For example, after upgrade when you try to bump the KRaft metadata version, you might get an error like this:

 

g.apache.kafka.common.errors.InvalidUpdateVersionException: Invalid update version 19 for feature metadata.version. Broker 3002 only supports versions 1-14 

In this case, 3002 is an old node that was removed before the upgrade and doesn't support the KRaft metadata version 19 and blocks the metadata update.

 

However, it seems to be impossible to list the registered nodes in order to unregister them:

• The describe cluster metadata request in the Admin API seems to return only the IDs of running brokers
• The describe metadata quorum command seems to list the removed nodes in the list of observers. But it does so only until the controller nodes are restarted.

If Kafka expects the inactive nodes to be registered, it should provide a list of the registered nodes so that it can be checked what nodes to unregister.