[KAFKA-15283] Client support for OffsetFetch and OffsetCommit with topic ID

• Type: Sub-task
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Fix Version/s: 4.0.0
• Component/s: clients, consumer
• Labels: kip-848-client-support newbie offset
• Assignee: Lianet Magrans
• Reporter: Kirk True
• Created: 01/Aug/23 00:48
• Updated: 27/Mar/24 18:20

Description

Currently, KafkaConsumer keeps track of topic IDs in the in-memory ConsumerMetadata object, and they are provided to the FETCH and METADATA RPC calls.

With KIP-848 the OffsetFetch and OffsetCommit will start using topic IDs in the same way, so the new client implementation will provide it when issuing those requests. Topic names should continue to be supported as needed by the AdminClient.

We should also review/clean-up the support for topic names in requests such as the METADATA request (currently supporting topic names as well as topic IDs on the client side).

Tasks include:

• Introduce Topic ID in existing OffsetFetch and OffsetCommit API that will be upgraded on the server to support topic ID
• Check topic ID propagation internally in the client based on RPCs including it.
• Review existing support for topic name for potential clean if not needed.

Issue Links

• is duplicated by: KAFKA-15325 Integrate topicId in OffsetFetch and OffsetCommit async consumer calls