Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.5, 6.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      Since LUCENE-6818 we have DFISimilarity which implements normalized chi-squared distance.

      But there are other alternatives (as described in http://trec.nist.gov/pubs/trec21/papers/irra.web.nb.pdf):

      • normalized chi-squared: "can be used for tasks that require high precision, against both short and long queries"
      • standardized: "good at tasks that require high recall and high precision, especially against short queries composed of a few words as in the case of Internet searches"
      • saturated: "for tasks that require high recall against long queries"

      I think we should just provide the three independence measures, and let the user choose. Similar to how we do DFR/IB/etc.

      1. LUCENE-6986.patch
        18 kB
        Robert Muir
      2. LUCENE-6986.patch
        18 kB
        Robert Muir
      3. LUCENE-6986.patch
        19 kB
        Robert Muir

        Activity

        Hide
        Robert Muir added a comment -

        patch.

        Show
        Robert Muir added a comment - patch.
        Hide
        Robert Muir added a comment -

        nukes unused DFR imports that got dragged in.

        Show
        Robert Muir added a comment - nukes unused DFR imports that got dragged in.
        Hide
        Robert Muir added a comment -

        more docs improvements and mark classes lucene.experimental.

        I think its ready.

        Show
        Robert Muir added a comment - more docs improvements and mark classes lucene.experimental. I think its ready.
        Hide
        ASF subversion and git services added a comment -

        Commit 1726205 from Robert Muir in branch 'dev/trunk'
        [ https://svn.apache.org/r1726205 ]

        LUCENE-6986: add more DFI measures

        Show
        ASF subversion and git services added a comment - Commit 1726205 from Robert Muir in branch 'dev/trunk' [ https://svn.apache.org/r1726205 ] LUCENE-6986 : add more DFI measures
        Hide
        ASF subversion and git services added a comment -

        Commit 1726212 from Robert Muir in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1726212 ]

        LUCENE-6986: add more DFI measures

        Show
        ASF subversion and git services added a comment - Commit 1726212 from Robert Muir in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1726212 ] LUCENE-6986 : add more DFI measures

          People

          • Assignee:
            Unassigned
            Reporter:
            Robert Muir
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development