There are actually three threads going on this ticket right now, there’s the “what threshold and expansion to use for geospatial” that I’d originally intended and provided a patch for, there’s the “what expansion for DocIdSetBuilder is generically optimal”, and there’s the “FBS is 50% of my allocation rate, can we pool” conversation.
I think the latter is a worthy conversation, and I don’t have a better place for it, so I’m going to continue to respond to the comments along those lines, (with apologies for the book I’m writing here) but I wanted to point out the divergence.
So, I certainly understand a knee-jerk reaction around using object pools of any kind. Yes, this IS what the JVM is for. It’s easier and simpler and lower maintenance to just use what’s provided. But I could also argue that Arrays.sort has all those same positive attributes and that hasn’t stopped several hand-written sort algorithms get into this codebase. The question is actually whether the easy and simple thing is good enough, or whether the harder thing has a sufficient offsetting benefit. Everyone on this thread is a highly experienced programmer, we all know this.
In this case, that means the question is actually whether the allocation rate is “good enough” or if there's a sufficiently offsetting opportunity for improvement, and arguments should ideally come from that analysis.
I can empirically state that for my large Solr index, that GC pause is the single biggest detriment to my 90+th percentile query latency. Put another way, Lucene is fantastically fast, at least when the JVM isn’t otherwise occupied. Because of shard fan-out, a per-shard p90 latency very quickly becomes a p50 latency for queries overall. (Even with mitigations like SOLR-4449)
I don’t think there’s anything particularly unique to my use-case in anything I just said, except possibly the word “large”.
As such, I consider this an opportunity for improvement, so I’ve suggested a mitigation strategy. It clearly has some costs. I’d be delighted to entertain any alternative strategies.
Actually, David Smiley did bring up one alternative suggestion for improvement, so let’s talk about -Xmn:
First, let’s assume that Lucene’s policy on G1 hasn’t changed, and we’re still talking about ParNew/CMS. Second, with the exception of a few things like cache, most of the allocations in a Solr/Lucene index are very short-lived. So it follows that given a young generation of sufficient size, the tenured generation would actually see very little activity.
The major disadvantage to just using a huge young generation then is that there aren’t any concurrent young-generation collectors. The bigger it is, the less frequently you need to collect, but the longer the stop-the-world GC pause when you do.
On the other end of the scale, a very small young space means shorter pauses, but far more frequent. Since almost all garbage is short-lived, maybe now you're doing young-collections so often that you’ve got the tenured collector doing a bunch of the work cleaning up short-lived objects too. (This can actually be a good thing, since the CMS collector is mostly concurrent)
There’s some theoretical size that optimizes frequency vs pause for averaged latency. Perhaps even by deliberately allowing some premature overflow into tenured simply because tenured can be collected concurrently. This kind of thing is extremely delicate to tune for though, especially since query rate (and query type distribution) can fluctuate. It’s easy to get it wrong, such that a sudden large-allocation slams past the rate CMS was expecting and triggers a full-heap stop-the-world pause.
I’m focusing on FBS here because: 1. Fifty Percent. 2. These are generally larger objects, so mitigating those allocations seemed like a good way to mitigate unexpected changes in allocation rate and allow more stable tuning.
There’s probably also at least one Jira issue around looking at object count allocation rate (vs size) since I suspect the single biggest factor in collector pause is the object count. Certainly I can point to objects that get allocated (by count) in orders of magnitude greater frequency than then next highest count. But since I don’t have a good an understanding of the use cases, let alone have any suggestions yet, I’ve left that for another time.