[KAFKA-12893] MM2 fails to replicate if starting two+ nodes same time

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 2.8.0
• Component/s: mirrormaker
• Assignee: Unassigned
• Reporter: Tommi Vainikainen
• Created: 04/Jun/21 13:34
• Updated: 05/Jul/23 14:51
• Resolved: 14/Feb/23 21:39

Description

I've observed a situation where starting more than one MM2 node in parallel, MM2 fails to start replication ie. replication flow seems to be stuck without action. I used exactly same mm2.properties file to start only one at a time, and the replication flow was proceeding smoothly.

In my setup dc1 has topic "mytopic1" and there is a producer with approx 1 msg/sec, and I'm trying to repilcate this to dc2. What I observed is that dc1.mytopic1 is created when initially launching two paraller MM2 instances, but no messages gets written into the topic as I would expect. If I kill MM2 instances, and only start one MM2 node, then MM2 starts replicating the messages in mytopic1.

My mm2.properties:

clusters=dc2, dc1
dc1->dc2.emit.heartbeats.enabled=true
dc1->dc2.enabled=true
dc1->dc2.sync.group.offsets.enabled=false
dc1->dc2.sync.group.offsets.interval.seconds=45
dc1->dc2.topics=mytopic1
dc1->dc2.topics.exclude=
dc1.bootstrap.servers=tvainika-dc1-dev-sandbox.aivencloud.com:12693
dc1.security.protocol=SSL
dc1.ssl.keystore.type=PKCS12
dc1.ssl.keystore.location=dc1/client.keystore.p12
dc1.ssl.keystore.password=secret
dc1.ssl.key.password=secret
dc1.ssl.truststore.location=dc1/client.truststore.jks
dc1.ssl.truststore.password=secret
dc2.bootstrap.servers=tvainika-dc2-dev-sandbox.aivencloud.com:12693
dc2.security.protocol=SSL
dc2.ssl.keystore.type=PKCS12
dc2.ssl.keystore.location=dc2/client.keystore.p12
dc2.ssl.keystore.password=secret
dc2.ssl.key.password=secret
dc2.ssl.truststore.location=dc2/client.truststore.jks
dc2.ssl.truststore.password=secret
tasks.max=3

Issue Links

• duplicates: KAFKA-9981 Running a dedicated mm2 cluster with more than one nodes,When the configuration is updated the task is not aware and will lose the update operation.
• is related to: KAFKA-10857 Mirror Maker 2 - replication not working when deploying multiple instances