The FastVectorHighlighter uses for every term found in a fragment an equal weight, which causes a higher ranking for fragments with a high number of words or, in the worst case, a high number of very common words than fragments that contains all of the terms used in the original query.
This patch provides ordered fragments with IDF-weighted terms:
total weight = total weight + IDF for unique term per fragment * boost of query;
The ranking-formula should be the same, or at least similar, to that one used in org.apache.lucene.search.highlight.QueryTermScorer.
The patch is simple, but it works for us.
- A better approach would be moving the whole fragments-scoring into a separate class.
- Switch scoring via parameter
- Exact phrases should be given a even better score, regardless if a phrase-query was executed or not
- edismax/dismax-parameters pf, ps and pf^boost should be observed and corresponding fragments should be ranked higher