Major Description
When I start a fresh cluster using bin/kafka-server-start.sh and issue a MetadataRequest to it, the results are blank. It's correct that there are no topics, but there are also no brokers returned. I'm writing a driver for Kafka, so this makes the initial connection to the cluster difficult.
To reproduce:
- Start Zookeeper with bin/zookeeper-server-start.sh config/zookeeper.properties and a broker with bin/kafka-server-start.sh config/server.properties. Be sure there's nothing in /tmp from a previous run.
- Run this echo -e "\x00\x00\x00\x15\x00\x03\x00\x01\x00\x00\x00\x00\x00\x07pykafka\x00\x00\x00\x00" | nc localhost 9092 | hd and observe the output:
00000000 00 00 00 0c 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000010 - Create a topic using bin/kafka-topics.sh --zookeeper localhost:2181 --create --topic test --partitions 2 --replication-factor 1
- Re-run the same command and now observe the output:
kfb@parsely-dev:~/src/ct/pykafka$ echo -e "\x00\x00\x00\x15\x00\x03\x00\x01\x00\x00\x00\x00\x00\x07pykafka\x00\x00\x00\x00" | nc localhost 9092 | hd 00000000 00 00 00 61 00 00 00 00 00 00 00 01 00 00 00 00 |...a............| 00000010 00 0b 70 61 72 73 65 6c 79 2d 64 65 76 00 00 23 |..parsely-dev..#| 00000020 84 00 00 00 01 00 00 00 04 74 65 73 74 00 00 00 |.........test...| 00000030 02 00 00 00 00 00 01 00 00 00 00 00 00 00 01 00 |................| 00000040 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00 00 |................| 00000050 00 00 00 00 00 00 00 00 01 00 00 00 00 00 00 00 |................| 00000060 01 00 00 00 00 |.....| 00000065
In this case, "parsely-dev" is the name of my work VM and the "#" following it is the port number. I've verified it's a correctly formatted MetadataResponse. It's the first null result that we've having a hard time dealing with.
As for the bytestring, that's a correctly formatted MetadataRequest with no topics specified. Presumably if I specified a topic name it would auto-create the topic and then start returning broker information. It doesn't really change the fact that the initial state is fairly broken.
Finally, it's worth noting that if I delete the "test" topic (after turning on delete.topic.enable) then the responses still include broker information. It's just the initial state which is causing problems.
kfb@parsely-dev:~/src/kafka$ bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic test Topic test is marked for deletion. Note: This will have no impact if delete.topic.enable is not set to true. kfb@parsely-dev:~/src/ct/pykafka$ echo -e "\x00\x00\x00\x15\x00\x03\x00\x01\x00\x00\x00\x00\x00\x07pykafka\x00\x00\x00\x00" | nc localhost 9092 | hd 00000000 00 00 00 21 00 00 00 00 00 00 00 01 00 00 00 00 |...!............| 00000010 00 0b 70 61 72 73 65 6c 79 2d 64 65 76 00 00 23 |..parsely-dev..#| 00000020 84 00 00 00 00 |.....| 00000025