[KAFKA-17210] Broker fixes for smooth concurrent fetches on share partition

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.9.0
• Assignee: Abhinav Dixit
• Reporter: Abhinav Dixit
• Created: 29/Jul/24 06:32
• Updated: 01/Sep/24 13:27
• Resolved: 30/Jul/24 07:16

Description

Identified a couple of reliability issues with broker code for share groups - 

1. Broker seems to get stuck at times when using multiple share consumers due to a corner case where the second last fetch request did not contain any topic partition to fetch, because of which the broker could never complete the last request. This results in a share fetch request getting stuck.
2. Since persister would not perform any business logic around sending state batches for a share partition, there could be scenarios where it sends state batches with no AVAILABLE records. This could cause a breach on the limit of in-flight messages we have configured, and hence broker would never be able to complete the share fetch requests.

Issue Links

• links to: GitHub Pull Request #16711