[LUCENE-2959] [GSoC] Implementing State of the Art Ranking for Lucene

• Type: New Feature
• Status: Reopened
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0-ALPHA, flexscoring branch
• Component/s: core/query/scoring, general/javadocs, modules/examples
• Labels: gsoc2011 lucene-gsoc-11 mentor
• Lucene Fields: New
• Assignee: Robert Muir
• Reporter: David Mark Nemeskey
• Created: 10/Mar/11 10:49
• Updated: 17/Sep/24 09:46
• Resolved: 11/Sep/11 15:49

Description

Lucene employs the Vector Space Model (VSM) to rank documents, which compares
unfavorably to state of the art algorithms, such as BM25. Moreover, the architecture is
tailored specically to VSM, which makes the addition of new ranking functions a non-
trivial task.

This project aims to bring state of the art ranking methods to Lucene and to implement a
query architecture with pluggable ranking functions.

The wiki page for the project can be found at http://wiki.apache.org/lucene-java/SummerOfCode2011ProjectRanking.

Issue Links

• is related to: LUCENE-6818 Implementing Divergence from Independence (DFI) Term-Weighting for Lucene/Solr
• relates to: LUCENE-2091 Add BM25 Scoring to Lucene
• relates to: LUCENE-2392 Enable flexible scoring

Sub-Tasks

• Rename some classes and methods in the search package so their names describe their roles: Open
• Similarity.Stats class for term & collection statistics: Resolved
• Implement various ranking models as Similarities: Resolved
• Unit and integration test cases for the new Similarities: Resolved
• EasySimilarity to interpret document length as float: Resolved
• Integrate MockBM25Similarity and MockLMSimilarity into the framework: Resolved
• Get javadoc for the similarities package in shape: Resolved
• Make EasySimilarityProvider a full-fledged class: Resolved
• Rename EasySimilarity to SimilarityBase: Resolved