We introduce KIP-500 to introduce a Self-Managed Metadata Quorum. We should always have 1 active controller, and all the RPC will send to the active controller. But there's chances that the active controller already changed, but the RPC still send to the old one.
In the attachment log, we can see:
[Controller 3002] Becoming active at controller epoch 1. ... [Controller 3000] Becoming active at controller epoch 2.
So, the latest active controller should be 3000. But the create topic RPC are all sending to controller 3002:
"errorMessage":"The active controller appears to be node 3000"
This bug causes the RaftClusterTests flaky.
Debug log while running testCreateClusterAndCreateListDeleteTopic test: https://drive.google.com/file/d/1WVUgy1Erjx8mHyofiP9MVvQGb0LcDYt3/view?usp=sharing
KAFKA-12629 Failing Test: RaftClusterTest
- links to