[STORM-2678] Improve performance of LoadAwareShuffleGrouping

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Component/s: storm-client
• Assignee: Jungtaek Lim
• Reporter: Jungtaek Lim
• Created: 04/Aug/17 04:19
• Updated: 07/Sep/17 18:10
• Resolved: 07/Sep/17 18:09

Description

While reviewing STORM-2306, I found a room to improve LoadAwareShuffleGrouping, basically taking similar approach with ShuffleGrouping.

The idea is constructing virtual ring (array) with proper length which tasks are randomly distributed to the ring respecting their loads, and just do sequential access like we do in ShuffleGrouping. Maybe slower to reflect load information due to construct virtual ring but no need to call randInt and additional loop for each tuple emission.

We may also want to reduce call of System.currentTimeMillis(), and if we get rid of that, it should be better.

Issue Links

• links to: GitHub Pull Request #2261