Major Description
I've noticed in a lot of clusters that creating new threads often takes a relatively long time, for whatever reason. Currently we create new threads while holding the threadpool's lock, which means that, even if other existing worker threads finish their current work items and are ready to handle newly enqueued work, everything is frozen while the thread is being created. This creates potentially long stalls resulting in increased queue time at the high percentiles.