[SAMZA-2785] Reuse of KafkaCheckpointManager causes memory leak

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Katie Liu
• Created: 22/Jun/23 21:13
• Updated: 22/Jun/23 21:13

Description

Calling start() and stop() multiple times on the same KafkaCheckpointManager, while stopConsumerAfterFirstRead == true, causes the SystemConsumer left unclosed. The unclosed SystemConsumer can cause memory leaks in some implementations.

 

Evidence:

In production logs, SystemConsumer was started 1741 times, but only closed 14 times.

[katxxxx@xxxxxx xxxx]$ grep "Starting the checkpoint SystemConsumer from oldest offset" xxxxxx.log|wc -l
1741
[katxxxxx@xxxxxx xxxx]$ grep "Stopping system consumer" xxxxxxxxxx.log |wc -l
14 

We also have a heap dump of KafkaSystemConsumer taking up 8Gbs of memory.