Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: 0.9.2-incubating
    • Fix Version/s: None
    • Component/s: storm-core
    • Labels:
      None

      Description

      In my profiling, I found the receiver thread of worker can be a performance bottle-neck.

      Now each worker has single receiver thread, and it is responsbile to transfer information generated from multiple netty client to tens of executor disruptor queue. It is too much for busy topology.

      Without this fix, we have to increase the number of workers, which will create more intra-worker traffic that we don't want.

      I suggest that we can add a config called "worker.receiver.thread.count" to control the parallism of the receiver thread, and make it default to 1.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                clockfly Sean Zhong
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: