Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-2590 KIP-28: Kafka Streams Checklist
  3. KAFKA-3455

Connect custom processors with the streams DSL

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Not A Problem
    • 0.10.0.1
    • None
    • streams

    Description

      From the kafka users email thread, we discussed the idea of connecting custom processors with topologies defined from the Streams DSL (and being able to sink data from the processor). Possibly this could involve exposing the underlying processor's name in the streams DSL so it can be connected with the standard processor API.

      Thanks for the feedback. This is definitely something we wanted to support
      in the Streams DSL.

      One tricky thing, though, is that some operations do not translate to a
      single processor, but a sub-graph of processors (think of a stream-stream
      join, which is translated to actually 5 processors for windowing / state
      queries / merging, each with a different internal name). So how to define
      the API to return the processor name needs some more thinking.

      Attachments

        Activity

          People

            Unassigned Unassigned
            jonathan.bender Jonathan Bender
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: