Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-2365 Copycat checklist
  3. KAFKA-2713

Copycat worker should not call connector's/task's start methods in the control thread

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.0.0
    • Component/s: KafkaConnect
    • Labels:
      None

      Description

      Currently the DistributedHerder calls start() methods in the same thread as the group membership is handled. This is simple and makes lifecycles easier to reason about, but also means that user code (and even code that can simply block for a long time, like sink task's connectors) can potentially block group membership, which in turn causes the worker to fall out of the group.

      To avoid this, we should run these methods in the worker thread for each connector/task.

        Attachments

          Activity

            People

            • Assignee:
              ewencp Ewen Cheslack-Postava
              Reporter:
              ewencp Ewen Cheslack-Postava
              Reviewer:
              Gwen Shapira
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: