[KAFKA-15859] Introduce delayed remote list offsets to make LIST_OFFSETS async

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0
• Assignee: Kamal Chandraprakash
• Reporter: Kamal Chandraprakash
• Created: 20/Nov/23 16:19
• Updated: 16/Oct/24 00:04
• Resolved: 15/Sep/24 23:25

Description

LIST_OFFSETS API request is handled by the request handler threads. If there are concurrent LIST_OFFSETS requests to remote storage more than the number of request handler threads, then other requests such as FETCH and PRODUCE might starve and be queued. This can lead to higher latency in producing/consuming messages.

The `offsetForTimes` call to remote storage can take time as it has to fetch the offset and time indexes to serve the request so moving the requests to purgatory and handle it via the remote-log-reader threads frees up the request handler threads to serve other requests.

Issue Links

• is a child of: KAFKA-16947 Kafka Tiered Storage V2
• links to: GitHub Pull Request #16320
• links to: GitHub Pull Request #16602
• links to: GitHub Pull Request #17045
• links to: GitHub Pull Request #17112
• links to: GitHub Pull Request #17487
• mentioned in: Page Loading...