[KAFKA-15305] The background thread should try to process the remaining task until the shutdown timer is expired

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.8.0
• Component/s: clients, consumer
• Labels: consumer-threading-refactor timeout
• Assignee: TengYao Chi
• Reporter: Philip Nee
• Created: 04/Aug/23 18:39
• Updated: 04/Jun/24 20:28
• Resolved: 04/Jun/24 20:28

Description

While working on https://issues.apache.org/jira/browse/KAFKA-15304

close() API supplies a timeout parameter so that the consumer can have a grace period to process things before shutting down.  The background thread currently doesn't do that, when close() is initiated, it will immediately close all of its dependencies.

 

This might not be desirable because there could be remaining tasks to be processed before closing.  Maybe the correct things to do is to first stop accepting API request, second, let the runOnce() continue to run before the shutdown timer expires, then we can force closing all of its dependencies.

Issue Links

• Blocked: KAFKA-16639 AsyncKafkaConsumer#close does not send heartbeat to leave group
• links to: GitHub Pull Request #16156