Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-2590 KIP-28: Kafka Streams Checklist
  3. KAFKA-3337

Extract selector as a separate groupBy operator for KTable aggregations

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.0.0
    • Fix Version/s: 0.10.0.0
    • Component/s: streams
    • Labels:

      Description

      Currently KTable aggregation takes a selector used for selecting the aggregate key.and an aggregator for aggregating the values with the same selected key, which makes the function a little bit "heavy":

      table.groupBy(initializer, adder, substractor, selector, /* optional serde*/);
      

      It is better to extract the selector in a separate groupBy function such that

      KTableGrouped KTable#groupBy(selector);
      
      KTable KTableGrouped#aggregate(initializer, adder, substractor, /* optional serde*/);
      

      Note that "KTableGrouped" only have APIs for aggregate and reduce, and none else. So users have to follow the pattern below:

      table.groupBy(...).aggregate(...);
      

      This pattern is more natural for users who are familiar with SQL / Pig or Spark DSL, etc.

        Attachments

          Activity

            People

            • Assignee:
              mjsax Matthias J. Sax
              Reporter:
              guozhang Guozhang Wang
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: