[KAFKA-15282] Implement client support for KIP-848 client-side assignors

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: clients, consumer
• Labels: kip-848-client-support
• Assignee: Unassigned
• Reporter: Kirk True
• Created: 01/Aug/23 00:39
• Updated: 27/Sep/24 11:32

Description

The client-side assignor provides the logic for the partition assignments instead of on the server. Client-side assignment is the main approach used by the “old protocol” for divvying up partitions. While the “new protocol” favors server-side assignors, the client-side assignor will continue to be used for backward compatibility, including KSQL, Connect, etc.

Note: I think that the client-side assignor logic and the reconciliation logic can remain separate from each other. We should strive to keep the two pieces unencumbered, unless it’s unavoidable.

This task includes:

• Validate the client’s configuration for assignor selection
• Integrate with the new PartitionAssignor interface to invoke the logic from the user-provided assignor implementation
• Implement the necessary logic around the request/response from the ConsumerGroupPrepareAssignment RPC call using the information from the PartitionAssignor above
• Implement the necessary logic around the request/response from the ConsumerGroupInstallAssignment RPC call, again using the information calculated by the PartitionAssignor

This task is part of the work to implement support for the new KIP-848 consumer group protocol.

Issue Links

• is related to: KAFKA-15280 Implement client support for selecting KIP-848 server-side assignor
• is related to: KAFKA-15279 Implement client support for KIP-848 client-side assigner RPCs