Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-2901

Extend ListGroups and DescribeGroup APIs to cover offsets

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 0.9.0.0
    • Fix Version/s: None
    • Component/s: consumer
    • Labels:
      None

      Description

      The ListGroupsRequest and DescribeGroupsRequest added to 0.9.0.0 allow admin tools to get details of consumer groups now that this information is not stored in ZK.

      The brokers also now store offset information for consumer groups. At present, there is no API for admin tools to discover the groups that brokers are storing offsets for.

      For example, if a consumer is using the new consumer api, is storing offsets in Kafka under the groupId 'Bob', but is using manual partition assignment, then the groupCache in the GroupMetadataManager will not contain any information about the group 'Bob'. However, the offsetCache in the GroupMetadataManager will contain information about 'Bob'.

      Currently the only way for admin tools to know the full set of groups being managed by Kafka, i.e. those storing offsets in Kafka, those using Kafka for balancing of consumer groups, and those using Kafka for both, is to consume the offset topic.

      We need to extend the List/Describe groups API to allow admin tools to discover 'Bob' and allow the partition offsets to be retrieved.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                hachikuji Jason Gustafson
                Reporter:
                BigAndy Andy Coates
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: