Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-355

Provide a SystemStreamPartitionGrouper that groups into N sets

    XMLWordPrintableJSON

    Details

    • Type: Task
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.8.0
    • Fix Version/s: None
    • Component/s: container
    • Labels:
      None

      Description

      As part of SAMZA-123, it was proposed to provide an SSPGrouper that would hash the SSPs into a fixed number of tasknames. This would provide similar, not-at-all-grouped functionality as the GroupByPartitionGrouper but with only n TaskInstances created rather than one for each SSP. If that n is tied to the number of containers, this is a conceptually simple way of processing lots of SSPs that need no co-grouping.

      As long as the n is not changed, the SSPs would be guaranteed to hash to the same TaskName with each run.

      There was some concern over if this is the correct approach to take, so it was agreed to post the patch separately and make it available to the community and for further discussion.

        Attachments

        1. SAMZA-355.patch
          5 kB
          Jakob Homan

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                jghoman Jakob Homan
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: