Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-7578

Kafka streams: add possibility to choose multiple output topics

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • None
    • None
    • streams

    Description

      KIP draft: https://cwiki.apache.org/confluence/display/KAFKA/KIP-422%3A+%5BStreams%5D+Support+multiple+topic+resolution+in+TopicNameExtractor

      There is an awesome feature which was added in 2.0 kafka stream - possibility to choose dynamically the output topic for topology, but in some cases it could be useful to chose several topics withing the same cluster.

      Personally me - I met such case: I needed to route message based on its content and by routes configuration to several topics.

      I've made a 'proposal' PR for this, unfortunately I couldn't find better way to implement this:

      https://github.com/apache/kafka/pull/5801

      If this approach is OK, and improvement could be done in future versions, please let me know and I'll finish PR code.

      Attachments

        Issue Links

        relates to

        Improvement - An improvement or enhancement to an existing feature or task. KAFKA-4936 Allow dynamic routing of output records

        • Major - Major loss of function.
        • Resolved

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            tdanylchuk Taras Danylchuk
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:

              Slack

                Issue deployment