[KAFKA-17077] the node.id is inconsistent to broker.id when "broker.id.generation.enable=true"

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 3.9.0
• Assignee: TengYao Chi
• Reporter: Chia-Ping Tsai
• Created: 03/Jul/24 19:27
• Updated: 19/Jul/24 02:41
• Resolved: 19/Jul/24 02:41

Description

We change the broker id of `KafkaConfig` directly when `broker.id.generation.enable=true` [0]. However, the update is NOT sync to node.id of `KafkaConfig`. It results in following issues:

1. we can see many "-1" in the log. for example:

[2024-07-03 19:23:08,453] INFO [ExpirationReaper--1-AlterAcls]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)

2. `KafkaRaftManager` will use uninitialized node.id to create `KafkaRaftClient` in migration [1], and the error sequentially happens

[0] https://github.com/apache/kafka/blob/27220d146c5d043da4adc3d636036bd6e7b112d2/core/src/main/scala/kafka/server/KafkaServer.scala#L261
[1] https://github.com/apache/kafka/blob/27220d146c5d043da4adc3d636036bd6e7b112d2/core/src/main/scala/kafka/raft/RaftManager.scala#L230

Issue Links

• causes: KAFKA-17128 Make node.id immutable after removing zookeeper migration
• links to: GitHub Pull Request #16540