[KAFKA-1479] Logs filling up while Kafka ReplicaFetcherThread tries to retrieve partition info for deleted topics

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 0.8.1
• Component/s: log
• Environment: CentOS
• Assignee: Unassigned
• Reporter: Manasi Manasi
• Created: 01/Jun/14 22:22
• Updated: 21/May/15 02:05
• Resolved: 17/Jul/14 18:39

Description

Started noticing that logs are filling up fast with lines like this:

[2014-06-01 15:18:08,218] WARN [KafkaApi-2] Fetch request with correlation id 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-27,26] failed due to Topic sams_2014-05-27 either doesn't exist or is in the process of being deleted (kafka.server.KafkaApis)
[2014-06-01 15:18:08,218] WARN [KafkaApi-2] Fetch request with correlation id 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-28,38] failed due to Topic sams_2014-05-28 either doesn't exist or is in the process of being deleted (kafka.server.KafkaApis)
[2014-06-01 15:18:08,219] WARN [KafkaApi-2] Fetch request with correlation id 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-30,20] failed due to Topic sams_2014-05-30 either doesn't exist or is in the process of being deleted (kafka.server.KafkaApis)
[2014-06-01 15:18:08,219] WARN [KafkaApi-2] Fetch request with correlation id 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-22,46] failed due to Topic sams_2014-05-22 either doesn't exist or is in the process of being deleted (kafka.server.KafkaApis)
[2014-06-01 15:18:08,219] WARN [KafkaApi-2] Fetch request with correlation id 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-27,8] failed due to Topic sams_2014-05-27 either doesn't exist or is in the process of being deleted (kafka.server.KafkaApis)

The above is from kafkaServer.out. Also seeing errors in server.log:

[2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for partition [sams_2014-05-26,19] to broker 0:class kafka.common.UnknownTopicOrPartitionException (kafka.server.ReplicaFetcherThread)
[2014-06-01 15:23:52,788] WARN [KafkaApi-2] Fetch request with correlation id 10887 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-30,4] failed due to Topic sams_2014-05-30 either doesn't exist or is in the process of being deleted (kafka.server.KafkaApis)
[2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for partition [sams_2014-05-24,34] to broker 0:class kafka.common.UnknownTopicOrPartitionException (kafka.server.ReplicaFetcherThread)
[2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for partition [sams_2014-05-26,41] to broker 0:class kafka.common.UnknownTopicOrPartitionException (kafka.server.ReplicaFetcherThread)
[2014-06-01 15:23:52,788] WARN [KafkaApi-2] Fetch request with correlation id 10887 from client ReplicaFetcherThread-0-2 on partition [2014-05-21,0] failed due to Topic 2014-05-21 either doesn't exist or is in the process of being deleted (kafka.server.KafkaApis)
[2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for partition [sams_2014-05-28,42] to broker 0:class kafka.common.UnknownTopicOrPartitionException (kafka.server.ReplicaFetcherThread)
[2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for partition [sams_2014-05-22,21] to broker 0:class kafka.common.UnknownTopicOrPartitionException (kafka.server.ReplicaFetcherThread)
[2014-06-01 15:23:52,788] WARN [KafkaApi-2] Fetch request with correlation id 10887 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-20,26] failed due to Topic sams_2014-05-20 either doesn't exist or is in the process of being deleted (kafka.server.KafkaApis)

All these partitions belong to deleted topics. Nothing changed on our end when we started noticing these logs filling up. Any ideas what is going on?