[KAFKA-16118] Coordinator unloading fails when replica is deleted

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.6.0
• Fix Version/s: 3.7.0
• Assignee: David Jacot
• Reporter: David Jacot
• Created: 12/Jan/24 09:43
• Updated: 16/Jan/24 07:31
• Resolved: 15/Jan/24 07:30

Description

The new group coordinator always expects the leader epoch to be received when it must unload the metadata for a partition. However, in KRaft, the leader epoch is not passed when the replica is delete (e.g. after reassignment).

java.lang.IllegalArgumentException: The leader epoch should always be provided in KRaft.
    at org.apache.kafka.coordinator.group.GroupCoordinatorService.onResignation(GroupCoordinatorService.java:931)
    at kafka.server.metadata.BrokerMetadataPublisher.$anonfun$onMetadataUpdate$9(BrokerMetadataPublisher.scala:200)
    at kafka.server.metadata.BrokerMetadataPublisher.$anonfun$onMetadataUpdate$9$adapted(BrokerMetadataPublisher.scala:200)
    at kafka.server.metadata.BrokerMetadataPublisher.$anonfun$updateCoordinator$4(BrokerMetadataPublisher.scala:397)
    at java.base/java.lang.Iterable.forEach(Iterable.java:75)
    at kafka.server.metadata.BrokerMetadataPublisher.updateCoordinator(BrokerMetadataPublisher.scala:396)
    at kafka.server.metadata.BrokerMetadataPublisher.$anonfun$onMetadataUpdate$7(BrokerMetadataPublisher.scala:200)
    at kafka.server.metadata.BrokerMetadataPublisher.onMetadataUpdate(BrokerMetadataPublisher.scala:186)
    at org.apache.kafka.image.loader.MetadataLoader.maybePublishMetadata(MetadataLoader.java:382)
    at org.apache.kafka.image.loader.MetadataBatchLoader.applyDeltaAndUpdate(MetadataBatchLoader.java:286)
    at org.apache.kafka.image.loader.MetadataBatchLoader.maybeFlushBatches(MetadataBatchLoader.java:222)
    at org.apache.kafka.image.loader.MetadataLoader.lambda$handleCommit$1(MetadataLoader.java:406)
    at org.apache.kafka.queue.KafkaEventQueue$EventContext.run(KafkaEventQueue.java:127)
    at org.apache.kafka.queue.KafkaEventQueue$EventHandler.handleEvents(KafkaEventQueue.java:210)
    at org.apache.kafka.queue.KafkaEventQueue$EventHandler.run(KafkaEventQueue.java:181)
    at java.base/java.lang.Thread.run(Thread.java:1583)
    at org.apache.kafka.common.utils.KafkaThread.run(KafkaThread.java:66)

The side effect of this bug is that group coordinator loading/unloading fails.

Issue Links

• links to: GitHub Pull Request #15182