[KAFKA-16648] Question: KIP-848 and KafkaTestKit.java

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: sanghyeok An
• Created: 30/Apr/24 10:25
• Updated: 30/Apr/24 10:26

Description

Hi, Kafka Team.
I am writing test code for the new rebalancing protocol proposed in KIP-848.

It works well in general code. However, it does not work properly when creating an EmbeddedBroker using KafkaTestKit.java.

 
 

Phenomena

1. Create a CombineBroker that acts as both controller and broker using KafkaTestKit.
2. Consumer do subscribe() and poll() to created Broker. 
    

At this time, the Consumer sends a HeartBeat Signal to the Broker successfully. However, it never receives a Partition Assigned response from the Broker.
 

What is my broker configs? 

 

Actual Broker Config.

I set controller.quorum.voters = 0@localhost:9093, but 0@0.0.0.0.0:0 is setted. 
Because of this codes (https://github.com/apache/kafka/blob/7c0a302c4da9d53a8fddc504a9fac8d8afecbec8/core/src/test/java/kafka/testkit/KafkaClusterTestKit.java#L305-L307)
 
 
 

My opinion.
I am not familiar with the broker's quorum, but it seems to be the problem.
 
I expect that when the Consumer sends a poll request to the broker, the group coordinator broker assigns the topic/partition and then performs quorum for each epoch number.
 
However, it seems to not work because the controller to vote is represented as 0.0.0.0:0.
 
This setting does not work well when applied to containers in docker-compose.
Could this be the cause of the problem?
 
 

Question
If controller.quorum.voters is set to 0.0.0.0:0 and i want to use consumer group rebalancing through KIP-848, what settings should be applied to the brokers and consumers?