Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-3487

KIP-146: Support per-connector/per-task classloaders in Connect

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 0.10.0.0
    • 0.11.0.0
    • connect

    Description

      Currently we just use the default ClassLoader in Connect. However, this limits how we can compatibly load conflicting connector plugins. Ideally we would use a separate class loader per connector/task that is instantiated to avoid potential conflicts.

      Note that this also opens up options for other ways to provide jars to instantiate connectors. For example, Spark uses this to dynamically publish classes defined in the REPL and load them via URL: https://ardoris.wordpress.com/2014/03/30/how-spark-does-class-loading/ But much simpler examples (include URL in the connector class instead of just class name) are also possible and could be a nice way to more support dynamic sets of connectors, multiple versions of the same connector, etc.

      Attachments

        Issue Links

          Activity

            People

              kkonstantine Konstantine Karantasis
              ewencp Ewen Cheslack-Postava
              Ewen Cheslack-Postava Ewen Cheslack-Postava
              Votes:
              2 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: