[KAFKA-13255] Mirrormaker config property config.properties.exclude is not working as expected

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.8.0
• Fix Version/s: 3.2.0, 3.1.1
• Component/s: mirrormaker
• Assignee: Ed Berezitsky
• Reporter: Anamika Nadkarni
• Created: 31/Aug/21 06:41
• Updated: 3 days ago 03:39
• Resolved: 15/Nov/21 10:47

Description

Objective - Use MM2 (kafka connect in distributed cluster) for data migration between cluster hosted in private data center and aws msk cluster.

Steps performed -

1. Started kafka-connect service.
2. Created 3 MM2 connectors (i.e. source connector, checkpoint connector and heartbeat connector). Curl commands used to create connectors are in the attached file.  To exclude certain config properties while topic replication, we are using the 'config.properties.exclude' property in the MM2 source connector.

Expected -

Source topic 'dev.portlandDc.anamika.helloMsk' should be successfully created in destination cluster.

Actual -

Creation of the source topic 'dev.portlandDc.anamika.helloMsk' in destination cluster fails with an error. Error is

[2021-08-06 06:13:40,944] WARN [mm2-msc|worker] Could not create topic dev.portlandDc.anamika.helloMsk. (org.apache.kafka.connect.mirror.MirrorSourceConnector:371)
org.apache.kafka.common.errors.InvalidConfigurationException: Unknown topic config name: confluent.value.schema.validation

Issue Links

• links to: GitHub Pull Request #11328
• links to: GitHub Pull Request #11401