Blocker Description
When initializing a KRaft cluster with SCRAM inter-broker authentication, you have to create user credentials using the StorageTool before starting the brokers:
bin/kafka-storage.sh format -c /opt/kafka/server3/config/server.properties" -t a2FdMvicQUmCYojQZnNsIw \
-S "SCRAM-SHA-512=[name=admin,password=changeit]"
This command should produce the following record in the metadata log:
| offset: 3 CreateTime: 1727435366178 keySize: -1 valueSize: 171 sequence: -1 headerKeys: [] payload: {"type":"USER_SCRAM_CREDENTIAL_RECORD","version":0,"data":{"name":"admin","mechanism":2,"salt":"bmNvZHNpNm1yaWdzbTcycndlcWJtdnltag==","storedKey":"00pZjSfcztrhNNgbP7VDwb22L+s8ySG+NfkF5+5AiytOdD/9gm2L7xxLkPO54lpF/sAD0mwcIm3rGWKqiIWdkg==","serverKey":"kQL0eg4cauRtKIhUf5zXK/3lLJe7TMRwcybUja7J49t3NJ5aM/o7lVm7RNbsxzhKxYqEAmRX6wjMkD8T7H6rxw==","iterations":4096}}
Then, at start time, the brokers would load these user credentials from metadata, and authenticate against each other, or clients presenting the same credentials.
It looks like this metadata record is not written anymore by the tool, so the authentication fails with invalid credentials because the user credentials cache is empty.
AFAICS, the issue was introduced here: https://github.com/apache/kafka/commit/02f541d4ea51ee9034f92d249dde96bc70860e5e.
Attachments
Issue Links
- links to