Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-2590 KIP-28: Kafka Streams Checklist
  3. KAFKA-2727

initialize only the part of the topology relevant to the task

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.0.0
    • Fix Version/s: 0.10.0.0
    • Component/s: streams
    • Labels:
      None

      Description

      Currently each streaming task initializes the entire topology regardless of the assigned topic-partitions. This is wasteful especially when the topology has local state stores. All local state stores are restored from their change log topics even when are not actually used in the task execution. To fix this, the task initialization should be aware of the relevant subgraph of the topology and initializes only processors and state stores in the subgraph.

        Attachments

          Activity

            People

            • Assignee:
              yasuhiro.matsuda Yasuhiro Matsuda
              Reporter:
              yasuhiro.matsuda Yasuhiro Matsuda
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: