Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.10.0.0
    • Fix Version/s: 0.10.1.0
    • Component/s: KafkaConnect
    • Labels:
      None

      Description

      While good for default configuration and reducing the total configuration the user needs to do, it's inconvenient requiring that all connectors on a cluster need to use the same converter. It's definitely a good idea to stay consistent, but occasionally you may need a special converters, e.g. one source of data happens to come in JSON despite you standardizing on Avro.

      Note that these configs are connector-level in the sense that the entire connector should use a single converter type, but since converters are used by tasks the config needs to be automatically propagated to tasks.

      This is effectively public API change as it is adding a built-in config for connectors/tasks, so this probably requires a KIP.

        Issue Links

          Activity

          Hide
          ASF GitHub Bot added a comment -

          GitHub user ewencp opened a pull request:

          https://github.com/apache/kafka/pull/1721

          KAFKA-3845: KIP-75: Add per-connector converters

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/ewencp/kafka kafka-3845-per-connector-converters

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/kafka/pull/1721.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1721


          commit d7763f995ed84df1af41bf99f96340a17ef051c5
          Author: Ewen Cheslack-Postava <me@ewencp.org>
          Date: 2016-08-12T03:32:57Z

          KAFKA-3845: KIP-75: Add per-connector converters


          Show
          ASF GitHub Bot added a comment - GitHub user ewencp opened a pull request: https://github.com/apache/kafka/pull/1721 KAFKA-3845 : KIP-75: Add per-connector converters You can merge this pull request into a Git repository by running: $ git pull https://github.com/ewencp/kafka kafka-3845-per-connector-converters Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/1721.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1721 commit d7763f995ed84df1af41bf99f96340a17ef051c5 Author: Ewen Cheslack-Postava <me@ewencp.org> Date: 2016-08-12T03:32:57Z KAFKA-3845 : KIP-75: Add per-connector converters
          Hide
          Gwen Shapira added a comment -

          Issue resolved by pull request 1721
          https://github.com/apache/kafka/pull/1721

          Show
          Gwen Shapira added a comment - Issue resolved by pull request 1721 https://github.com/apache/kafka/pull/1721
          Hide
          ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/kafka/pull/1721

          Show
          ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1721

            People

            • Assignee:
              Ewen Cheslack-Postava
              Reporter:
              Ewen Cheslack-Postava
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development