Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-6687

Allow to read a topic multiple times

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0
    • Component/s: streams
    • Labels:
      None

      Description

      Users often want to read topic multiple times. However, this is not possible because there is a single consumer and thus a topic can only be consumed once.

      Users get an exception

      Exception in thread “main” org.apache.kafka.streams.errors.TopologyException: Invalid topology: Topic source has already been registered by another source.

      If they use a topic name in multiple `stream()`, `table()`, `globalTable()` calls.

      However, with KAFKA-6034 in place, we could allow adding a topic multiple times and rewrite the topology internally to only read the topic once. This would simplify application code as users don't need to put workaround in place to get the same behavior.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ableegoldman A. Sophie Blee-Goldman
                Reporter:
                mjsax Matthias J. Sax
              • Votes:
                8 Vote for this issue
                Watchers:
                15 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: