[KAFKA-17608] KRaft controller crashes when leader is removed from quorum - ASF JIRA

Attach files

Attach Screenshot

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Blocker
Resolution: Fixed
Affects Version/s: 3.9.0
Fix Version/s: 3.9.0
Component/s: controller
Labels:
None

Description

After removing the active controller with:

bin/kafka-metadata-quorum.sh --command-config config/kraft/controller2.properties --bootstrap-controller localhost:9093 remove-controller --controller-id 1 --controller-directory-id XvLXb0JPTSKRfsy9t47l-Q
Removed  KRaft controller 1 with directory id XvLXb0JPTSKRfsy9t47l-Q

The active controller fails with:

[2024-09-25 17:35:35,287] ERROR Encountered fatal fault: exception while renouncing leadership (org.apache.kafka.server.fault.ProcessTerminatingFaultHandler)
java.lang.RuntimeException: Unable to reset to last stable offset 11. No in-memory snapshot found for this offset.
        at org.apache.kafka.controller.OffsetControlManager.deactivate(OffsetControlManager.java:268)
        at org.apache.kafka.controller.QuorumController.renounce(QuorumController.java:1281)
        at org.apache.kafka.controller.QuorumController$QuorumMetaLogListener.lambda$handleLeaderChange$2(QuorumController.java:1177)
        at org.apache.kafka.controller.QuorumController$QuorumMetaLogListener.lambda$appendRaftEvent$3(QuorumController.java:1201)
        at org.apache.kafka.controller.QuorumController$ControllerEvent.run(QuorumController.java:585)
        at org.apache.kafka.queue.KafkaEventQueue$EventContext.run(KafkaEventQueue.java:132)
        at org.apache.kafka.queue.KafkaEventQueue$EventHandler.handleEvents(KafkaEventQueue.java:215)
        at org.apache.kafka.queue.KafkaEventQueue$EventHandler.run(KafkaEventQueue.java:186)
        at java.base/java.lang.Thread.run(Thread.java:840)