Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-2590 KIP-28: Kafka Streams Checklist
  3. KAFKA-3455

Connect custom processors with the streams DSL

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Not A Problem
    • Affects Version/s: 0.10.0.1
    • Fix Version/s: None
    • Component/s: streams
    • Labels:

      Description

      From the kafka users email thread, we discussed the idea of connecting custom processors with topologies defined from the Streams DSL (and being able to sink data from the processor). Possibly this could involve exposing the underlying processor's name in the streams DSL so it can be connected with the standard processor API.

      Thanks for the feedback. This is definitely something we wanted to support
      in the Streams DSL.

      One tricky thing, though, is that some operations do not translate to a
      single processor, but a sub-graph of processors (think of a stream-stream
      join, which is translated to actually 5 processors for windowing / state
      queries / merging, each with a different internal name). So how to define
      the API to return the processor name needs some more thinking.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              jonathan.bender Jonathan Bender
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: