I just read through all the comments to get up to speed on the history and issues involved.
There is definitely more complexity for generic per-segment visiting, and trying to support nifty multiple-dispatch by visitor method argument type.
For per-segment rewriting, arguably Query's existing rewrite method is fine. Not complicating this issue with per-segment support would help – no changes to IndexSearcher. I think (but am not certain) the use-case of rewriting the query for caching identical sub-clauses could be done by a wrapping query that implements rewrite() to then call the API we're talking about here to share some state and then propagate the rewrite() call.
The question is if we should really commit this without any specific usecase or other code using it :/
I expect that a good query visitor (or query "processor" as you all seem to want to call it, despite the "visit" method terminology) would eliminate the ugly massive query branching logic found in these classes:
- MultiTermHighlighting (org.apache.lucene.search.postingshighlight)
- FieldQuery (org.apache.lucene.search.vectorhighlight)
- WeightedSpanTermExtractor (org.apache.lucene.search.highlight)
It's no surprise to me that all of those points are in the highlighting code. I too want to rewrite a query to make it highlightable. My query has custom span query subclasses that the branches listed above do not know about. The custom query wraps another query (or queries) that these branches do know about. There is no per-segment concern here.
I've also once wrote a query rewriter that was not related to highlighting – it rewrote a wildcard query to use an ngram tokenized field. That code was annoying, verbose, and didn't support Queries I had never seen before. There is no per-segment concern here either.
As a separate issue, I think making Query immutable (and demanding subclasses do the same) is a good idea for Lucene 5.
Does anyone have any opinions on what I had to say? I may have some time soon to continue where Chris left off. According to the last comment, Uwe hoped to produce a fix for the sortability of generic methods. If that doesn't materialize, I think it's far simpler to have a generic Query argument to the visit interface as was originally proposed. In summary I think we can make this simple.