[SOLR-8183] Add Learning to Rank framework to Solr

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Ajinkya Kale
• Created: 21/Oct/15 18:39
• Updated: 16/Jan/16 12:25

Description

Add a Learning to Rank or Machine Learned Ranking framework to Solr. The framework should be able to
1. Generate user-result-click tracking logs
2. Learning/Training component which is based on something like Ranklib (http://sourceforge.net/p/lemur/wiki/RankLib/)
3. Push the model learnt in step 2 to Solr through the existing rerank framework

I have a very crude experimental code for step 2 and 3 working (thanks to the help from Joel Bernstein).
For step 1 I was wondering if Solr has an open source version of https://docs.lucidworks.com/display/lweug/Click+Scoring+Relevance+Framework
If there existing something like this then it wont be hard to close the loop and have an nice feedback loop implementation on LTR framework.

I feel there should be a big push on the overall algorithmic front which will help LTR framework.
Something like the Fair Pairs algorithm switch in Solr to turn on/off to generate unbiased clicked data will be pretty hand for optimizing the ranking through LTR.
http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/36363.pdf
http://www.cs.cornell.edu/people/tj/publications/radlinski_joachims_06a.pdf
Also better support for experimenting with multi-arm bandits in search rankings will be pretty handy too! https://www.cs.cornell.edu/~rdk/papers/icml08.pdf

This calls for a better architecture to
1. Collect user behavior data
2. Learn a ML-model based on User data
3. Experiment / collect unbiased datasets from deployed models

Issue Links

• is related to: SOLR-8542 Integrate Learning to Rank into Solr