[GEARPUMP-52] support local shuffling

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: streaming
• Assignee: Unassigned
• Reporter: Manu Zhang
• Created: 20/Apr/16 05:33
• Updated: 20/Apr/16 05:33

Description

imported from https://github.com/gearpump/gearpump/issues/1649.

Storm has a grouping strategy called LocalOrShuffleGrouping which will have better performance than ShuffleGrouping in theory.

Local or shuffle grouping: If the target bolt has one or more tasks in the same worker process, tuples will be shuffled to just those in-process tasks. Otherwise, this acts like a normal shuffle grouping.

This requires context information of tasks in the same Executor.