Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-1569

Allowing users to specify the nimbus thrift server queue size.

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.0
    • Fix Version/s: 2.0.0
    • Component/s: storm-core
    • Labels:
      None

      Description

      Currently the nimbus sever in secure mode uses https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ThreadPoolExecutor.html Backed by https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/SynchronousQueue.html, Please see https://github.com/apache/thrift/blob/0.9.2/lib/java/src/org/apache/thrift/server/TThreadPoolServer.java#L132. This means that if all executor threads are busy serving a request and new requests come in we will see RejectedExecutionExceptions in logs once they have reached the retry limit. Instead we should allow the requests to be queued. This patch allows the requests to be queued by replacing SynchronousQueue with https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ArrayBlockingQueue.html with default size of 100000 requests which should be large enough for most applications. Applications can modify this default by adding the config nimbus.queue.size to their storm.yaml and bouncing nimbus.

        Attachments

          Activity

            People

            • Assignee:
              parth.brahmbhatt Parth Brahmbhatt
              Reporter:
              parth.brahmbhatt Parth Brahmbhatt
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: