[KAFKA-17788] During ZK migration, always include control.plane.listener.name in advertisedBrokerListeners

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 3.9.0
• Fix Version/s: 3.9.0
• Assignee: Colin McCabe
• Reporter: Jakub Scholz
• Reviewer: Luke Chen
• Created: 14/Oct/24 09:49
• Updated: 15/Oct/24 21:35
• Resolved: 15/Oct/24 21:35

Description

When testing migration with Kafka 3.9.0-RC2, the broker fail to start when they are first rolled to start the migration with the following error:

2024-10-11 21:37:04,060 ERROR Exiting Kafka due to fatal exception (kafka.Kafka$) [main]
java.lang.IllegalArgumentException: requirement failed: control.plane.listener.name must be a listener name defined in advertised.listeners. The valid options based on currently configured listeners are REPLICATION-9091,PLAIN-9092,TLS-9093
	at scala.Predef$.require(Predef.scala:337)
	at kafka.server.KafkaConfig.validateValues(KafkaConfig.scala:1019)
	at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:843)
	at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:185)
	at kafka.Kafka$.buildServer(Kafka.scala:71)
	at kafka.Kafka$.main(Kafka.scala:90)
	at kafka.Kafka.main(Kafka.scala)

This is despite our configuration having the

{control.plane.listener.name}

properly configured:

listener.security.protocol.map=CONTROLPLANE-9090:SSL,REPLICATION-9091:SSL,PLAIN-9092:SASL_PLAINTEXT,TLS-9093:SSL
listeners=CONTROLPLANE-9090://0.0.0.0:9090,REPLICATION-9091://0.0.0.0:9091,PLAIN-9092://0.0.0.0:9092,TLS-9093://0.0.0.0:9093
advertised.listeners=CONTROLPLANE-9090://cluster-9833cba6-kafka-1.cluster-9833cba6-kafka-brokers.test-suite-namespace.svc:9090,REPLICATION-9091://cluster-9833cba6-kafka-1.cluster-9833cba6-kafka-brokers.test-suite-namespace.svc:9091,PLAIN-9092://cluster-9833cba6-kafka-1.cluster-9833cba6-kafka-brokers.test-suite-namespace.svc:9092,TLS-9093://cluster-9833cba6-kafka-1.cluster-9833cba6-kafka-brokers.test-suite-namespace.svc:9093
inter.broker.listener.name=REPLICATION-9091
control.plane.listener.name=CONTROLPLANE-9090

It looks like 3.9.0-RC2 filters out the control plane listener (maybe because it is used by the KRaft controllers as well?) and runs into this error. This worked fine in 3.8.0, so this seems like a regression in 3.9.0 that should be fixed.

The full log from the broker node is attached. It includes the full configuration of the broker as well.

Issue Links

• links to: GitHub Pull Request #17506