[KAFKA-3845] Support per-connector converters - ASF JIRA

Attach files

Attach Screenshot

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Resolved
Priority: Critical
Resolution: Fixed
Affects Version/s: 0.10.0.0
Fix Version/s: 0.10.1.0
Component/s: KafkaConnect
Labels:
None

Description

While good for default configuration and reducing the total configuration the user needs to do, it's inconvenient requiring that all connectors on a cluster need to use the same converter. It's definitely a good idea to stay consistent, but occasionally you may need a special converters, e.g. one source of data happens to come in JSON despite you standardizing on Avro.

Note that these configs are connector-level in the sense that the entire connector should use a single converter type, but since converters are used by tasks the config needs to be automatically propagated to tasks.

This is effectively public API change as it is adding a built-in config for connectors/tasks, so this probably requires a KIP.