[KAFKA-17439] Make polling for new records an explicit action/event in the new consumer

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.7.0
• Fix Version/s: 4.0.0
• Component/s: clients, consumer
• Labels: consumer-threading-refactor kip-848-client-support
• Assignee: Kirk True
• Reporter: Kirk True
• Created: 28/Aug/24 19:20
• Updated: 26/Sep/24 02:29

Description

Presently, the new consumer polls the FetchRequestManager many, many times a second and creates fetch requests for any fetchable partitions. In order to more closely mirror how the existing consumer processes fetches, we should mirror the points at which fetch requests are sent in the new consumer.

Issue Links

• is related to: KAFKA-17182 Consumer fetch sessions are evicted too quickly
• links to: GitHub Pull Request #17035