[KAFKA-16985] Ensure consumer attempts to send leave request on close even if interrupted

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.8.0
• Fix Version/s: 4.0.0
• Component/s: clients, consumer
• Labels: kip-848-client-support
• Assignee: Kirk True
• Reporter: Lianet Magrans
• Reviewer: Lianet Magrans
• Created: 17/Jun/24 17:09
• Updated: 15/Nov/24 01:44
• Resolved: 13/Nov/24 19:27

Description

While running some stress tests we found out consumers were not leaving the group if interrupted while closing (leading to members getting eventually fenced after the session expired). On close, the consumer generates an Unsubscribe event to be handled in the background, but we noticed that the network thread failed with the interruption, seemingly not sending the unsent requests. We should review this to ensure that a member does a clean leave, notifying the coordinator with a leave HB, even if in a fire-and-forget mode in the case of interruption (and validate the legacy consumer behaviour in this scenario).  (Still under investigation, I'll update more info as I discover it)

Issue Links

• fixes: KAFKA-17518 AsyncKafkaConsumer cannot reliably leave group when closed with small timeout
• fixes: KAFKA-17519 Define and validate correctness of Consumer.close() and its timeout when thread is interrupted
• relates to: KAFKA-17518 AsyncKafkaConsumer cannot reliably leave group when closed with small timeout
• relates to: KAFKA-17519 Define and validate correctness of Consumer.close() and its timeout when thread is interrupted
• links to: GitHub Pull Request #16686