Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-2687

Add support for ListGroups and DescribeGroup APIs

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.0.0
    • Component/s: None
    • Labels:
      None

      Description

      Since the new consumer currently has no persistence in Zookeeper (pending outcome of KAFKA-2017), there is no way for administrators to investigate group status including getting the list of members in the group and their partition assignments. We therefore propose to modify GroupMetadataRequest (previously known as ConsumerMetadataRequest) to return group metadata when received by the respective group's coordinator. When received by another broker, the request will be handled as before: by only returning coordinator host and port information.

      GroupMetadataRequest => GroupId IncludeMetadata
        GroupId => String
        IncludeMetadata => Boolean
      
      GroupMetadataResponse => ErrorCode Coordinator GroupMetadata
        ErrorCode => int16
        Coordinator => Id Host Port
          Id => int32
          Host => string
          Port => int32
        GroupMetadata => State ProtocolType Generation Protocol Leader  Members
          State => String
          ProtocolType => String
          Generation => int32
          Protocol => String
          Leader => String
          Members => [Member MemberMetadata MemberAssignment]
            Member => MemberIp ClientId
              MemberIp => String
              ClientId => String
            MemberMetadata => Bytes
            MemberAssignment => Bytes
      

      The request schema includes a flag to indicate whether metadata is needed, which saves clients from having to read all group metadata when they are just trying to find the coordinator. This is important to reduce group overhead for use cases which involve a large number of topic subscriptions (e.g. mirror maker).

      Tools will use the protocol type to determine how to parse metadata. For example, when the protocolType is "consumer", the tool can use ConsumerProtocol to parse the member metadata as topic subscriptions and partition assignments.

      The detailed proposal can be found below.
      https://cwiki.apache.org/confluence/display/KAFKA/KIP-40%3A+ListGroups+and+DescribeGroup

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                hachikuji Jason Gustafson
                Reporter:
                hachikuji Jason Gustafson
                Reviewer:
                Jun Rao
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: