[KAFKA-15948] Refactor AsyncKafkaConsumer shutdown

• Type: Improvement
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 3.8.0
• Component/s: clients, consumer
• Labels: consumer-threading-refactor
• Assignee: Philip Nee
• Reporter: Philip Nee
• Created: 29/Nov/23 19:28
• Updated: 02/Feb/24 22:21
• Resolved: 28/Dec/23 04:07

Description

Upon closing we need a round trip from the network thread to the application thread and then back to the network thread to complete the callback invocation.  Currently, we don't have any of that.  I think we need to refactor our closing mechanism.  There are a few points to the refactor:

1. The network thread should know if there's a custom user callback to trigger or not.  If there is, it should wait for the callback completion to send a leave group.  If not, it should proceed with the shutdown.
2. The application thread sends a closing signal to the network thread and continuously polls the background event handler until time runs out.