Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
Kill all zookeeper and kafka process. Clear zookeeper and kafka data dir. Restart zookeeper and kafka. If there are any active client. Topic used by client will be auto-created.
How to reproduce?
1. Start zookeeper and kafka zookeeper and kafka config file.
nohup bin/zookeeper-server-start.sh config/zookeeper.properties &
nohup bin/kafka-server-start.sh config/server.properties &
2. Create topic test with 2 partitions
bin/kafka-topics.sh --create --zookeeper localhost:2181 --topic test --partitions 2 --replication-factor 1
3. Produce some data to topic test
bin/kafka-producer-perf-test.sh --topic test --num-records 50000000 --record-size 100 --throughput=-1 --producer-props bootstrap.servers=localhost:9092
4. Kill zookeeper and kafka. ProducerPerformance is still running.
jps
21072 QuorumPeerMain
21704 ProducerPerformance
21230 Kafka
21854 Jps
kill -9 21072 21230
5. Remove Zookeeper and Kafka data
rm -rf /tmp/zookeeper/
rm -rf /tmp/kafka-logs/
6. Start zookeeper and kafka
nohup bin/zookeeper-server-start.sh config/zookeeper.properties &
nohup bin/kafka-server-start.sh config/server.properties &
7. Check topic and you’ll see there is topic named test with partition 1. And the ProducerPerformance process continues to run normally.
bin/kafka-topics.sh --describe --zookeeper localhost:2181
Topic: test PartitionCount: 1 ReplicationFactor: 1 Configs:
Topic: test Partition: 0 Leader: 0 Replicas: 0 Isr: 0
Some output of ProducerPerformance process.
1995632 records sent, 399126.4 records/sec (38.06 MB/sec), 378.6 ms avg latency, 435.0 ms max latency.
org.apache.kafka.common.errors.TimeoutException: Expiring 148 record(s) for test-1:120000 ms has passed since batch creation
………………………..
org.apache.kafka.common.errors.TimeoutException: Expiring 148 record(s) for test-1:121774 ms has passed since batch creation
1711254 records sent, 342250.8 records/sec (32.64 MB/sec), 2324.5 ms avg latency, 123473.0 ms max latency.