Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-14843

Connector plugins config endpoint does not include Common configs

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.2.0, 3.3.0, 3.2.1, 3.4.0, 3.2.2, 3.2.3, 3.3.1, 3.3.2
    • 3.2.4, 3.5.0, 3.4.1, 3.3.3
    • connect
    • None

    Description

      Connector plugins GET config endpoint introduced in https://cwiki.apache.org/confluence/display/KAFKA/KIP-769%3A+Connect+APIs+to+list+all+connector+plugins+and+retrieve+their+configuration+definitions  allows to get plugin configuration from the rest endpoint.

      This configuration only includes the plugin configuration, but not the base configuration of the Sink/Source Connector.

      For instance, when validating the configuration of a plugin, all configs are returned:

      ```

      curl -s $CONNECT_URL/connector-plugins/io.aiven.kafka.connect.http.HttpSinkConnector/config | jq -r '.[].name' | sort -u | wc -l     
      21

      curl -s $CONNECT_URL/connector-plugins/io.aiven.kafka.connect.http.HttpSinkConnector/config/validate -XPUT -H 'Content-type: application/json' --data "{\"connector.class\": \"io.aiven.kafka.connect.http.HttpSinkConnector\", \"topics\": \"example-topic-name\"}" | jq -r '.configs[].definition.name' | sort -u | wc -l
      39

      ```

      and the missing configs are all from base config:

      ```

      diff validate.txt config.txt                                                                                                    
      6,14d5
      < config.action.reload
      < connector.class
      < errors.deadletterqueue.context.headers.enable
      < errors.deadletterqueue.topic.name
      < errors.deadletterqueue.topic.replication.factor
      < errors.log.enable
      < errors.log.include.messages
      < errors.retry.delay.max.ms
      < errors.retry.timeout
      16d6
      < header.converter
      24d13
      < key.converter
      26d14
      < name
      33d20
      < predicates
      35,39d21
      < tasks.max
      < topics
      < topics.regex
      < transforms
      < value.converter

      ```

      Would be great to get the base configs from the same endpoint as well, so we could rely on it instead of using the validate endpoint to get all configs.

      Attachments

        Issue Links

          Activity

            People

              jeqo Jorge Esteban Quilcate Otoya
              jeqo Jorge Esteban Quilcate Otoya
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: