[KAFKA-13628] Consider to provide a API to get the latest offset commits without respect to a specific group

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Labels: need-kip needs-kip
• Assignee: Unassigned
• Reporter: Luke Chen
• Created: 30/Jan/22 09:17
• Updated: 30/Jan/22 09:17

Description

context: KAFKA-13612.

 

Currently, getting the lag of Kafka Consumer groups need a lot of calls:

• poll every broker in the cluster with ListGroupsRequests
• send FindCoordinator requests for every group and put them in batches
• send DescribeGroupRequest to each coordinator with the batches of consumer groups
• read the results out of the DescribeGroupResponses and go round the loop again

 

There is a gap in the consumer administrative APIs and we should try to address it with an API that can actually offer compatibility guarantees. For example, perhaps we could introduce an OffsetQuery API or something like that to make it easier to get the latest offset commits without respect to a specific group.