[KAFKA-9830] DeadLetterQueueReporter leaks KafkaProducer instance

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.3.2, 2.6.0, 2.4.2, 2.5.1
• Component/s: connect
• Assignee: Greg Harris
• Reporter: Greg Harris
• Created: 07/Apr/20 17:22
• Updated: 27/May/20 19:27
• Resolved: 30/Apr/20 02:42

Description

The DeadLetterQueueReporter (introduced by KAFKA-6738) creates a KafkaProducer to report errors to Kafka, but does not clean up the producer, leaving many idle network threads open after tasks are stopped.

Reproduction steps:
Start a task that has a non-empty DLQ topic name
Stop the task
Observe the list of running threads

Expected result:
There is no thread related to the stopped task's DLQ left running

Actual result:
There is a thread named something like kafka-producer-network-thread | connector-dlq-producer-task-0" #1234 left running

Issue Links

• is duplicated by: KAFKA-10045 Kafka Connect Connection leak to Kafka Brokers when using dead letter queue
• links to: GitHub Pull Request #8442
• links to: GitHub Pull Request #8594