Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-13253

Kafka Connect losing task (re)configuration when connector name has special characters

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 2.7.1
    • None
    • KafkaConnect
    • None

    Description

      When not leader, DistributedHerder.reconfigureConnector() forwards the task configuration to the leader as follow:

      String reconfigUrl = RestServer.urlJoin(leaderUrl, "/connectors/" + connName + "/tasks");

      log.trace("Forwarding task configurations for connector {} to leader", connName);

      RestClient.httpRequest(reconfigUrl, "POST", null, rawTaskProps, null, config, sessionKey, requestSignatureAlgorithm);

      The problem is that if the connector name contains some special characters, such as '<', '>'... they need to be 'URLEncoded' appropriately, otherwise an uncatched exception is raised in RestClient and the forward is lost.

      Here is the kind of exception we can catch by adding the necessary code in RestClient:

      java.lang.IllegalArgumentException: Illegal character in path at index 51: http://10.224.0.15:8083/connectors/mirror1-cluster->mirror2-cluster.MirrorHeartbeatConnector/tasks

      An additional catch() should be added in RestClient.httpRequest(), here:

      catch (IOException | InterruptedException | TimeoutException | ExecutionException e) {
          log.error("IO error forwarding REST request: ", e);
          throw new ConnectRestException(Response.Status.INTERNAL_SERVER_ERROR, "IO Error trying to forward REST request: " + e.getMessage(), e);

      } finally {

      to catch all other exceptions because without, this kind of problem is completly silent.

      To reproduce:

      • start 2 kafka clusters
      • start a kafka connect (distributed) with at least 2 nodes
      • start an HeartbeatConnector with name "cluster1->cluster2"

      If the node which generated the task is not the leader (not systematic), it will forward the creation to the leader and it will be lost. As a result, the connector will stay in RUNNING state but without any task.

      Problem not easy to reproduce, it is important to start with empty connect topics to reproduce more easily

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              ddufour1a David Dufour
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: