Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-6182

Automatic co-partitioning of topics via automatic intermediate topic with matching partitions

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: None
    • Component/s: streams
    • Labels:
      None

      Description

      Currently it is up to the user to ensure that two input topics for a join have the same number of partitions, and if they don't, manually create an intermediate topic, and send the stream #through that topic first, and then performing the join.
      It would be great to have Kafka streams detect this and at least give the user the option to create an intermediate topic automatically with the same number of partitions as the topic being joined with.

      See https://docs.confluent.io/current/streams/developer-guide.html#joins-require-co-partitioning-of-the-input-data

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                astubbs Antony Stubbs
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: