[STORM-272] Make worker receiver thread number configurable

• Type: Sub-task
• Status: Closed
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: 0.9.2-incubating
• Component/s: storm-core
• Assignee: Unassigned
• Reporter: Sean Zhong
• Created: 01/Apr/14 13:43
• Updated: 09/Oct/15 01:29
• Resolved: 09/Oct/15 01:29

Description

In my profiling, I found the receiver thread of worker can be a performance bottle-neck.

Now each worker has single receiver thread, and it is responsbile to transfer information generated from multiple netty client to tens of executor disruptor queue. It is too much for busy topology.

Without this fix, we have to increase the number of workers, which will create more intra-worker traffic that we don't want.

I suggest that we can add a config called "worker.receiver.thread.count" to control the parallism of the receiver thread, and make it default to 1.

Issue Links

• is duplicated by: STORM-297 Storm Performance cannot be scaled up by adding more CPU cores