[STORM-1657] A new strategy in RAS that allows one worker to run executors from one component

• Type: Story
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Assignee: Unassigned
• Reporter: Zhuo Liu
• Created: 25/Mar/16 19:14
• Updated: 03/Jan/20 16:17
• Resolved: 03/Jan/20 16:17

Description

There are still lots of complaints from users and industry saying that Storm is hard to debug, mainly because Storm runs executors from different components (spout, bolt) in a single JVM process.
The original motivation for doing that is to allow inter-component communication to happen within a single process rather than over network socket as much as possible.

For small topology, this seems make sense, where we can place executors even within one or several worker. But for larger topology with more executors in each component, the saving in network transfer by using round-robin executor distribution to different workers is marginal. And the trade-off is that we will lose lots of debuggability, which makes problem-cracking difficult and prevents some potential users to start using Storm.

With RAS (STORM-894), it should be convenient for us to add a new strategy to allow one worker to only run executors (>= 1) from a single component, which will increase a few workers but not waste any memory /CPU resources.
By using this strategy (per-topology basis), it will be easier for users to find which component in a topology is having problems by checking their worker log, gc log and heap dump, without being bothered by noise made by other components' instances.

Issue Links

• duplicates: STORM-3482 Implement One Worker Per Component Option
• relates to: STORM-3480 Implement One Executor Per Worker RAS Option