[FLINK-9287] KafkaProducer011 seems to leak threads when not in exactly-once mode

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.4.2, 1.5.0
• Fix Version/s: 1.5.0
• Component/s: Connectors / Kafka
• Assignee: Piotr Nowojski
• Reporter: Christopher Ng
• Created: 02/May/18 10:41
• Updated: 07/May/18 07:50
• Resolved: 07/May/18 06:29

Description

KafkaProducer011 appears to be leaking kafka-producer-network-thread threads.  As far as I can tell it happens when it is not in EXACTLY_ONCE mode, it seems that it creates a FlinkKafkaProducer but never closes it, even when the FlinkKafkaProducer011 itself is closed.

I observed this when running a local cluster and submitting and then cancelling a job, a lot of kafka threads were left alive afterwards.

Issue Links

• links to: GitHub Pull Request #5952