Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-9615

Refactor TaskManager to extract task creation / cleanup out of StreamThread

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: streams
    • Labels:
      None

      Description

      We have a TODO marker for moving the task-creators into the task-manager as a follow-up of the tech cleanup, and here are some rationales:

      1. right now the reason we keep the task-creators in stream-thread is to be able to mock task creation in task-manager tests, but that should be better achieved with easy mocks on interfaces than this.

      2. the thread only need the thread-producer for a) metrics exposure and b) thread-metadata's producer-client ids. Both of them can be exposed from the task-manager instead of thread-producer.

      So the idea is that we let the task-manager to abstract / mock the task-creation and manage the producer(s) internally as well, and expose to stream-thread only the metrics / client-ids information upon request. With that it makes more sense to let the creator be the closer of the producer(s) — i.e. the task-manager, not the stream-producer.

      And we can fix a couple "FIXMEs" as well along with this effort.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                vvcephei John Roesler
                Reporter:
                guozhang Guozhang Wang
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: