[KAFKA-3845] Support per-connector converters

• Type: Improvement
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.10.0.0
• Fix Version/s: 0.10.1.0
• Component/s: KafkaConnect
• Assignee: Ewen Cheslack-Postava
• Reporter: Ewen Cheslack-Postava
• Created: 15/Jun/16 18:11
• Updated: 12/Sep/16 23:56
• Resolved: 19/Aug/16 03:57

Description

While good for default configuration and reducing the total configuration the user needs to do, it's inconvenient requiring that all connectors on a cluster need to use the same converter. It's definitely a good idea to stay consistent, but occasionally you may need a special converters, e.g. one source of data happens to come in JSON despite you standardizing on Avro.

Note that these configs are connector-level in the sense that the entire connector should use a single converter type, but since converters are used by tasks the config needs to be automatically propagated to tasks.

This is effectively public API change as it is adding a built-in config for connectors/tasks, so this probably requires a KIP.

Issue Links

• links to: GitHub Pull Request #1721
• mentioned in: Page Loading...