[SOLR-16675] Introduce the possibility to rerank topK results with vector similarity functions using DenseVectorField - ASF JIRA

Attach files

Attach Screenshot

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Task
Status: Closed
Priority: Blocker
Resolution: Done
Affects Version/s: None
Fix Version/s: 9.3
Component/s: None
Labels:
- vector-based-search

Description

When using knnQParser in reranking pay attention to the top-K parameter.

The second pass score(deriving from KNN search) is calculated only if the document d from the first pass is within the K nearest neighbors(in the whole index) of the target vector to search.

This is a current limitation.

The final ranked list of results will have the first pass score(main query q) combined with the second pass score(the approximated similarity function distance to the target vector to search).

Ideally, it should be possible to:

Rerank top K results with vector similarity. We should compute the vector similarity function using the DenseVectorField value of all the documents in top K results without the need of running a KNN query.
Use only the second pass score as the final score