[KAFKA-12244] Deleting a topic within metadata.max.idle after last message floods log with warnings

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 2.5.0
• Component/s: producer
• Environment: Linux, Confluent 5.5.0
• Assignee: Unassigned
• Reporter: Bart van Deenen
• Created: 27/Jan/21 09:35
• Updated: 27/Jan/21 09:35

Description

In a test we produce to a topic, then stop producing and delete the topic ca. 30 seconds later.

This leads to a flood of WARN messages (ca. 10 per second) after several minutes that lasts for several minutes.

WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-2] Error while fetching metadata with correlation id 141 : {test-topic=UNKNOWN_TOPIC_OR_PARTITION}

Investigation has shown that the issue can be solved by setting the metadata.max.idle.ms property to shorter than the interval between stopping messages and deleting the topic.

The issue itself is not critical, but can lead to a large pollution of your log, and thereby obscuring (or possibly losing) important messages.