[KAFKA-14838] MM2 Worker/Connector/Task clients should specify client ID based on flow and role

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Done
• Fix Version/s: 3.5.0
• Assignee: Daniel Urban
• Reporter: Daniel Urban
• Created: 23/Mar/23 14:55
• Updated: 31/Mar/23 15:44
• Resolved: 31/Mar/23 14:50

Description

MM2 code creates a lot of Kafka clients internally. These clients generate a lot of logs, but since the client.id is not properly specified, connecting the dots between a specific Connector/Task and its internal client is close to impossible. This is even more complex when MM2 is running in distributed mode, in which multiple Connect workers are running inside the same process.

For Connector/Task created clients, the client.id  clients should specify the flow, the Connector name/Task ID and the role of the client. E.g. MirrorSourceConnector uses multiple admin clients, and their client.id should reflect the difference between them.

For Worker created clients, the client.id should refer to the flow.

This will help log analysis significantly, especially in MM2 mode.

Issue Links

• links to: GitHub Pull Request #13458