[SAMZA-355] Provide a SystemStreamPartitionGrouper that groups into N sets

• Type: Task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.8.0
• Component/s: container
• Assignee: Unassigned
• Reporter: Jakob Homan
• Created: 26/Jul/14 02:06
• Updated: 26/Jul/14 02:08

Description

As part of SAMZA-123, it was proposed to provide an SSPGrouper that would hash the SSPs into a fixed number of tasknames. This would provide similar, not-at-all-grouped functionality as the GroupByPartitionGrouper but with only n TaskInstances created rather than one for each SSP. If that n is tied to the number of containers, this is a conceptually simple way of processing lots of SSPs that need no co-grouping.

As long as the n is not changed, the SSPs would be guaranteed to hash to the same TaskName with each run.

There was some concern over if this is the correct approach to take, so it was agreed to post the patch separately and make it available to the community and for further discussion.

Issue Links

• depends upon: SAMZA-123 Move topic partition grouping to the AM and generalize