[KAFKA-14843] Connector plugins config endpoint does not include Common configs - ASF JIRA

Agile Board

Attach files

Attach Screenshot

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 3.2.0, 3.3.0, 3.2.1, 3.4.0, 3.2.2, 3.2.3, 3.3.1, 3.3.2
Fix Version/s: 3.2.4, 3.5.0, 3.4.1, 3.3.3
Component/s: connect
Labels:
None

Description

Connector plugins GET config endpoint introduced in https://cwiki.apache.org/confluence/display/KAFKA/KIP-769%3A+Connect+APIs+to+list+all+connector+plugins+and+retrieve+their+configuration+definitions allows to get plugin configuration from the rest endpoint.

This configuration only includes the plugin configuration, but not the base configuration of the Sink/Source Connector.

For instance, when validating the configuration of a plugin, all configs are returned:

```

curl -s $CONNECT_URL/connector-plugins/io.aiven.kafka.connect.http.HttpSinkConnector/config | jq -r '.[].name' | sort -u | wc -l
21

curl -s $CONNECT_URL/connector-plugins/io.aiven.kafka.connect.http.HttpSinkConnector/config/validate -XPUT -H 'Content-type: application/json' --data "{\"connector.class\": \"io.aiven.kafka.connect.http.HttpSinkConnector\", \"topics\": \"example-topic-name\"}" | jq -r '.configs[].definition.name' | sort -u | wc -l
39

```

and the missing configs are all from base config:

```

diff validate.txt config.txt
6,14d5
< config.action.reload
< connector.class
< errors.deadletterqueue.context.headers.enable
< errors.deadletterqueue.topic.name
< errors.deadletterqueue.topic.replication.factor
< errors.log.enable
< errors.log.include.messages
< errors.retry.delay.max.ms
< errors.retry.timeout
16d6
< header.converter
24d13
< key.converter
26d14
< name
33d20
< predicates
35,39d21
< tasks.max
< topics
< topics.regex
< transforms
< value.converter

```

Would be great to get the base configs from the same endpoint as well, so we could rely on it instead of using the validate endpoint to get all configs.