Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-2798

Kafka Connect distributed configs can conflict with producer/consumer configs, making it impossible to control them independently

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.0.0
    • Component/s: KafkaConnect
    • Labels:
      None

      Description

      Right now we're using the approach to configs used by serializers where we just forward the configs for the entire worker to the producer and consumer. However, with the distributed mode, we now have a lot of conflicting configs because the distributed herder uses the group membership functionality and client networking libraries.

      We should instead use a sort of "namespaced" set of configs, which we already started doing with converters since we have multiple instances of converters instantiated by a single worker. We can use producer. and consumer. prefixes to isolate settings for the different components.

        Attachments

          Activity

            People

            • Assignee:
              ewencp Ewen Cheslack-Postava
              Reporter:
              ewencp Ewen Cheslack-Postava
              Reviewer:
              Gwen Shapira
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: