but it's trickier since the precStep is final (maybe we can un-final it and add a setter?)
Please don't do this. It is maybe better to do it like in Elasticsearch: Have a pool of NTS for each precision step.
this optimization has proven to help a lot in the context of ES, but we can use a static thread local since we are fully in control of the threading model. With Lucene itself, where it can be used in many different environment, then this can cause some unexpected behavior. For example, this might cause Tomcat to warn on leaking resources when unloading a war.
Thanks Shay: This is really the reason why we always refused to use static ThreadLocals in Lucene, especially for those heavy used components.
Maybe we can do a similar thing like with StringField in Mike's patch. Its a bit crazy to move out the TokenStreams from the field, but we can do this for performance here. Just have a lazy init pool of NumericTokenStreams for each precisionStep in each per thread DocumentsWriter (DefaultIndexingChain).
-1 to add thread locals in Lucene here!
Another idea how to manage the pools: Maybe add a protected method to Field that can get the DocumentsWriter instance and add some caching functionality for arbitrary TokenStreams (not just NumericTS or StringTS): Maybe some method on the per thread DocumentsWriter to set aTokenStream for reuse per field. The field (also custom ones) then could use setCachedTokenStream/getCachedTokenStream through the DocumentsWriter accessor from inside the Field.