Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-13731

Standalone Connect workers should not require connector configs to start

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 3.4.0
    • connect
    • None

    Description

      In order to start a standalone Connect worker, it's currently necessary to provide at least two command-line arguments:

      1. The path to the worker config file
      2. One or more paths to a connector config file

      This may be due to the now-inaccurate belief that standalone workers do not support the Connect REST API, which can be used to create/reconfigure/delete connectors at runtime. However, standalone mode does in fact expose the Connect REST API, and since that allows for connectors to be created after bringing up the worker, it is unnecessary to force users to supply at least one config for a connector to be run on the worker at startup time.

      We should remove the requirement that at least one connector config file be provided to standalone Connect workers at startup time. This has several benefits:

      1. Easier to debug for developers investigating framework behavior, including potential bugs
      2. Easier to experiment against for developers/developer evangelists who want to write quickstarts that leverage standalone mode
      3. Allows for connector config files to be tracked exclusively in JSON format (the REST API uses JSON for connector configs, whereas the configs passed to standalone workers on startup have to be Java properties files)
      4. Helps avoid perpetuating the unfortunately common myth that standalone workers do not expose a rest API
      5. Allows for standalone workers to be brought up even if there are issues with connector configs (if there are validation or startup issues with connector configs provided on the command line, the worker will fail to start up)

      Attachments

        Issue Links

          Activity

            People

              ChrisEgerton Chris Egerton
              ChrisEgerton Chris Egerton
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: