Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-6387

Worker's producer and consumer configs should inherit from worker configs

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: 1.0.0
    • Fix Version/s: None
    • Component/s: KafkaConnect
    • Labels:

      Description

      Currently, the worker configuration file defines the connection properties for the three separate types of connections being made to the Kafka cluster:

      1. the worker group membership,
      2. producers for source connectors,
      3. the consumers for sink connectors.

      The configs are namespaced because to properly support things like interceptors where the configs for 2 and 3 would conflict (same config name, different value).

      However, it would be beneficial when such control is not required for the producers and consumers to inherit the top-level configurations yet be able to override them with the producer. and consumer. namespaced configurations. This way the producer- and consumer-specific configurations need only be specified if/when they need to override the top-level configurations. This may be necessary, for example, to have different ACLs than the connector tasks compared to the producers and consumers.

      This will require a minimal KIP to explain the new behavior.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              rhauch Randall Hauch
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: