[KAFKA-12150] Consumer group refresh not working with clustered MM2 setup

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 2.7.0
• Component/s: mirrormaker
• Assignee: Unassigned
• Reporter: Ara Zarifian
• Created: 06/Jan/21 03:06
• Updated: 14/Feb/23 21:39
• Resolved: 14/Feb/23 21:39

Description

I'm running MM2 with Kafka 2.7 with the following configuration:

clusters = eastus2, westus
eastus2.bootstrap.servers = clusrter1.example.com:9092
westus.bootstrap.servers = cluster2.example.com:9092
eastus2->westus.enabled = true
eastus2->westus.topics = .*
westus->eastus2.enabled = true
westus->eastus2.topics = .*
refresh.topics.enabled = true
refresh.topics.interval.seconds = 5
refresh.groups.enabled = true
refresh.groups.interval.seconds = 5
sync.topic.configs.enabled = true
sync.topic.configs.interval.seconds = 5 
sync.topic.acls.enabled = false
sync.topic.acls.interval.seconds = 5
sync.group.offsets.enabled = true
sync.group.offsets.interval.seconds = 5
emit.checkpoints.enabled = true
emit.checkpoints.interval.seconds = 5
emit.heartbeats.enabled = true
emit.heartbeats.interval.seconds = 5
replication.factor = 3
checkpoints.topic.replication.factor = 3
heartbeats.topic.replication.factor = 3
offset-syncs.topic.replication.factor = 3
offset.storage.replication.factor = 3
status.storage.replication.factor = 3
config.storage.replication.factor = 3

More specifically, I'm running multiple instances of MM2 with the above configuration within Kubernetes pods. I was testing the new automatic consumer group offset translation functionality and noticed what appears to be a problem when running more than 1 instance of MM2 in this fashion.

Based on the KEP, I should be able to run multiple instances in this manner (see "Running a dedicated MirrorMaker cluster"), however, I noticed that when enabling replication using a 3-instance MM2 cluster, consumer groups were not synchronizing across clusters at all.

When running through my test case with a single MM2 instance, consumer group synchronization appears to work as expected consistently. When running through my 3-node test case, synchronization begins as soon as I scale the number of replicas to 1.

Am I misinterpreting the manner in which the KEP describes MM2 clusters or is this interaction an unexpected one?

Issue Links

• duplicates: KAFKA-9981 Running a dedicated mm2 cluster with more than one nodes,When the configuration is updated the task is not aware and will lose the update operation.
• is related to: KAFKA-10857 Mirror Maker 2 - replication not working when deploying multiple instances