[KAFKA-2798] Kafka Connect distributed configs can conflict with producer/consumer configs, making it impossible to control them independently

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.9.0.0
• Component/s: KafkaConnect
• Assignee: Ewen Cheslack-Postava
• Reporter: Ewen Cheslack-Postava
• Reviewer: Gwen Shapira
• Created: 10/Nov/15 18:23
• Updated: 10/Nov/15 19:08
• Resolved: 10/Nov/15 19:08

Description

Right now we're using the approach to configs used by serializers where we just forward the configs for the entire worker to the producer and consumer. However, with the distributed mode, we now have a lot of conflicting configs because the distributed herder uses the group membership functionality and client networking libraries.

We should instead use a sort of "namespaced" set of configs, which we already started doing with converters since we have multiple instances of converters instantiated by a single worker. We can use producer. and consumer. prefixes to isolate settings for the different components.