Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-2590 KIP-28: Kafka Streams Checklist
  3. KAFKA-3477

Add customizable StreamPartition into #to functions of Streams DSL

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.10.0.0
    • 0.10.0.0
    • streams

    Description

      In the lower-level Processor API we allow users to pass in a customizable StreamPartitioner when creating a new sink processor node to the topology:

      builder.addSink(String name, String topic, StreamPartitioner partitioner, String... parentNames));
      

      This StreamPartitioner allows users to specify any partitioning schemes based on record values instead of using the default behavior of hashing on the message key; but it is not exposed in the higher-level Streams DSL.

      We can add this parameter to the Streams DSL as well:

      KStream#to(String topic, StreamPartitioner partitioner);
      KTable#to(String topic, StreamPartitioner partitioner);
      

      Attachments

        Activity

          People

            mjsax Matthias J. Sax
            guozhang Guozhang Wang
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: