Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-10721

Rewrite topology to allow for overlapping unequal topic subscriptions

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: In Progress
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: streams
    • Labels:

      Description

      Minor followup improvement to KAFKA-6687 in which we rewrite the topology to make it possible for a user to subscribe multiple KStream/KTables to the same topic or identical set of topics. We could further extend this to make it possible for multiple KStream/KTables to be subscribed to overlapping but not identical sets of topics, ie

      KStream streamA = builder.stream("topic");
      KStream streamB = builder.stream("topic", "other-topic"); 

      One way to do this would be to break up multiple-topic source nodes into multiple single-topic sources that get merged together in the child node.

      See https://github.com/apache/kafka/pull/9582/files#diff-ac1bf2b23b80784dec20b00fdc42f2df7e5a5133d6c68978fa44aea11e950c3aR347-R349

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jsalavatr Salavat Zainullin
                Reporter:
                ableegoldman A. Sophie Blee-Goldman
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: