[KAFKA-9981] Running a dedicated mm2 cluster with more than one nodes,When the configuration is updated the task is not aware and will lose the update operation.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.4.0, 2.5.0, 2.4.1
• Fix Version/s: 3.5.0
• Component/s: mirrormaker
• Assignee: Chris Egerton
• Reporter: victor
• Created: 12/May/20 15:15
• Updated: 14/Feb/23 21:39
• Resolved: 09/Feb/23 15:50

Description

DistributedHerder.reconfigureConnector induction config update as follows:

if (changed) {
    List<Map<String, String>> rawTaskProps = reverseTransform(connName, configState, taskProps);
    if (isLeader()) {
        configBackingStore.putTaskConfigs(connName, rawTaskProps);
        cb.onCompletion(null, null);
    } else {
        // We cannot forward the request on the same thread because this reconfiguration can happen as a result of connector
        // addition or removal. If we blocked waiting for the response from leader, we may be kicked out of the worker group.
        forwardRequestExecutor.submit(new Runnable() {
            @Override
            public void run() {
                try {
                    String leaderUrl = leaderUrl();
                    if (leaderUrl == null || leaderUrl.trim().isEmpty()) {
                        cb.onCompletion(new ConnectException("Request to leader to " +
                                "reconfigure connector tasks failed " +
                                "because the URL of the leader's REST interface is empty!"), null);
                        return;
                    }
                    String reconfigUrl = RestServer.urlJoin(leaderUrl, "/connectors/" + connName + "/tasks");
                    log.trace("Forwarding task configurations for connector {} to leader", connName);
                    RestClient.httpRequest(reconfigUrl, "POST", null, rawTaskProps, null, config, sessionKey, requestSignatureAlgorithm);
                    cb.onCompletion(null, null);
                } catch (ConnectException e) {
                    log.error("Request to leader to reconfigure connector tasks failed", e);
                    cb.onCompletion(e, null);
                }
            }
        });
    }
}

KafkaConfigBackingStore task checks for configuration updates,such as topic whitelist update.If KafkaConfigBackingStore task is not running on leader node,an HTTP request will be send to notify the leader of the configuration update.However,dedicated mm2 cluster does not have the HTTP server turned on,so the request will fail to be sent,causing the update operation to be lost.

Issue Links

• is duplicated by: KAFKA-10857 Mirror Maker 2 - replication not working when deploying multiple instances
• is duplicated by: KAFKA-12150 Consumer group refresh not working with clustered MM2 setup
• is duplicated by: KAFKA-12893 MM2 fails to replicate if starting two+ nodes same time
• is duplicated by: KAFKA-13808 Mirrormaker2 stop sync data when modify topic partition in "Running a dedicated MirrorMaker cluster" mode
• is fixed by: KAFKA-10586 Full support for distributed mode in dedicated MirrorMaker 2.0 clusters
• links to: GitHub Pull Request #8656
• links to: GitHub Pull Request #12016
• links to: GitHub Pull Request #13137