[KAFKA-16639] AsyncKafkaConsumer#close does not send heartbeat to leave group

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 3.8.0
• Component/s: clients, consumer
• Labels: kip-848-client-support
• Assignee: TengYao Chi
• Reporter: Chia-Ping Tsai
• Created: 28/Apr/24 17:09
• Updated: 31/May/24 20:23
• Resolved: 31/May/24 20:14

Description

This bug can be reproduced by immediately closing a consumer which is just created.

The root cause is that we skip the new heartbeat used to leave group when there is a in-flight heartbeat request (https://github.com/apache/kafka/blob/5de5d967adffd864bad3ec729760a430253abf38/clients/src/main/java/org/apache/kafka/clients/consumer/internals/HeartbeatRequestManager.java#L212).

It seems to me the simple solution is that we create a heartbeat request when meeting above situation and then send it by pollOnClose (https://github.com/apache/kafka/blob/5de5d967adffd864bad3ec729760a430253abf38/clients/src/main/java/org/apache/kafka/clients/consumer/internals/RequestManager.java#L62).

 

Issue Links

• Blocked: KAFKA-15305 The background thread should try to process the remaining task until the shutdown timer is expired
• links to: GitHub Pull Request #16017