Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-2590 KIP-28: Kafka Streams Checklist
  3. KAFKA-3311

Move co-partition checking to PartitionAssignor and auto-create internal topics

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.0.0
    • Component/s: None
    • Labels:
      None

      Description

      Right now the internal topics management (i.e. re-partition topics for now) is buggy such that it was not auto-created by the assignor. We need to fix this by:

      1) moving co-paritition info into the assignor.
      2) let assignor create the internal topics with the right number of partitions as: if co-partitioned, equal to the other partitions; otherwise if there are other source topics in the sub-topology, equal to the maximum of other partitions; otherwise equal to the number of tasks writing to this topic.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                guozhang Guozhang Wang
                Reporter:
                guozhang Guozhang Wang
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: