[KAFKA-13401] Introduce a new Interface to manage Kafka resources in MM2

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.4.0
• Component/s: mirrormaker
• Assignee: Omnia Ibrahim
• Reporter: Omnia Ibrahim
• Created: 25/Oct/21 17:19
• Updated: 13/Sep/23 20:24
• Resolved: 15/Nov/22 10:21

Description

MM2 uses AdminClients to create and modify topics; while this simplifies the implementation of MM2 it has the following problems 

• It assumes that whoever runs MM2 must have admin access to both source and destination clusters. This assumption is not necessarily valid all the time.
• It creates conflict in use-cases where centralised systems or tools manage Kafka resources. 

It would be easier if customers could provide how they want to manage Kafka topics and configs through admin client or using their centralised system or tools. 

Issue Links

• causes: KAFKA-14443 Mirror Maker Connectors leak admin clients used for topic creation
• causes: KAFKA-15465 MM2 not working when its internal topics are pre-created on a cluster that disallows topic creation
• links to: GitHub Pull Request #12577