Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-8183

Add Learning to Rank framework to Solr

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Add a Learning to Rank or Machine Learned Ranking framework to Solr. The framework should be able to
      1. Generate user-result-click tracking logs
      2. Learning/Training component which is based on something like Ranklib (http://sourceforge.net/p/lemur/wiki/RankLib/)
      3. Push the model learnt in step 2 to Solr through the existing rerank framework

      I have a very crude experimental code for step 2 and 3 working (thanks to the help from Joel Bernstein).
      For step 1 I was wondering if Solr has an open source version of https://docs.lucidworks.com/display/lweug/Click+Scoring+Relevance+Framework
      If there existing something like this then it wont be hard to close the loop and have an nice feedback loop implementation on LTR framework.

      I feel there should be a big push on the overall algorithmic front which will help LTR framework.
      Something like the Fair Pairs algorithm switch in Solr to turn on/off to generate unbiased clicked data will be pretty hand for optimizing the ranking through LTR.
      http://static.googleusercontent.com/media/research.google.com/en//pubs/archive/36363.pdf
      http://www.cs.cornell.edu/people/tj/publications/radlinski_joachims_06a.pdf
      Also better support for experimenting with multi-arm bandits in search rankings will be pretty handy too! https://www.cs.cornell.edu/~rdk/papers/icml08.pdf

      This calls for a better architecture to
      1. Collect user behavior data
      2. Learn a ML-model based on User data
      3. Experiment / collect unbiased datasets from deployed models

        Issue Links

          Activity

          Hide
          ajinkyakale Ajinkya Kale added a comment -

          Here is a hacky piece I wrote in a few hours https://github.com/ajkl/solr-ltr-plugin
          Its not polished but it works.

          I would also like to mention that I could write such a functionality without knowing Solr internals and codebase is a big win for Solr and its plugin architecture! Thank you all who made it possible.

          Show
          ajinkyakale Ajinkya Kale added a comment - Here is a hacky piece I wrote in a few hours https://github.com/ajkl/solr-ltr-plugin Its not polished but it works. I would also like to mention that I could write such a functionality without knowing Solr internals and codebase is a big win for Solr and its plugin architecture! Thank you all who made it possible.
          Hide
          joel.bernstein Joel Bernstein added a comment -

          Hi Ajinkya Kale,

          Can you attach your code prototype? This will help people understand your implementation ideas.

          Show
          joel.bernstein Joel Bernstein added a comment - Hi Ajinkya Kale , Can you attach your code prototype? This will help people understand your implementation ideas.

            People

            • Assignee:
              Unassigned
              Reporter:
              ajinkyakale Ajinkya Kale
            • Votes:
              3 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:

                Development