Lucene - Core
  1. Lucene - Core
  2. LUCENE-6939

BlendedInfixSuggester to support exponential reciprocal BlenderType

    Details

    • Lucene Fields:
      New

      Description

      The orignal BlendedInfixSuggester introduced in LUCENE-5354 has support for:

      • BlenderType.POSITION_LINEAR and
      • BlenderType.POSITION_RECIPROCAL .

      These are used to score documents based on the position of the matched token i.e the closer is the matched term to the beginning, the higher score you get.

      In some use cases, we need a more aggressive scoring based on the position.
      That's where the exponential reciprocal comes into play
      i.e
      coef = 1/Math.pow(position+1, exponent)
      where the exponent is a configurable variable.

      1. LUCENE-6939.patch
        14 kB
        Arcadius Ahouansou

        Activity

        Hide
        Arcadius Ahouansou added a comment -

        Hello Michael McCandless
        Would you mind having a look at this initial patch when you have the chance?

        I am keen to making changes if needed especially regarding the backward compatibility part of the BlenderType.

        Thank you very much.

        Show
        Arcadius Ahouansou added a comment - Hello Michael McCandless Would you mind having a look at this initial patch when you have the chance? I am keen to making changes if needed especially regarding the backward compatibility part of the BlenderType. Thank you very much.
        Hide
        Michael McCandless added a comment -

        Thanks Arcadius Ahouansou, the patch looks good, except this is an experimental API so we don't need back-compat here (don't need to add LINEAR and RECIPROCAL. Also, instead of adding a new constructor, can you just add the new parameter to the existing "expert" constructor?

        Show
        Michael McCandless added a comment - Thanks Arcadius Ahouansou , the patch looks good, except this is an experimental API so we don't need back-compat here (don't need to add LINEAR and RECIPROCAL . Also, instead of adding a new constructor, can you just add the new parameter to the existing "expert" constructor?
        Hide
        Arcadius Ahouansou added a comment -

        Thank you very much Michael McCandless for your comments.

        I have now

        • removed the enums (added for backward compatibility) and
        • re-used the existing constructor instead.

        Thank you very much.

        Show
        Arcadius Ahouansou added a comment - Thank you very much Michael McCandless for your comments. I have now removed the enums (added for backward compatibility) and re-used the existing constructor instead. Thank you very much.
        Hide
        Michael McCandless added a comment -

        Thanks Arcadius Ahouansou, patch looks good! I'll commit soon ... but looks like svn is down at the moment.

        Show
        Michael McCandless added a comment - Thanks Arcadius Ahouansou , patch looks good! I'll commit soon ... but looks like svn is down at the moment.
        Hide
        ASF subversion and git services added a comment -

        Commit 1721330 from Michael McCandless in branch 'dev/trunk'
        [ https://svn.apache.org/r1721330 ]

        LUCENE-6939: add exponential reciprocal scoring mode to BlendedInfixSuggester

        Show
        ASF subversion and git services added a comment - Commit 1721330 from Michael McCandless in branch 'dev/trunk' [ https://svn.apache.org/r1721330 ] LUCENE-6939 : add exponential reciprocal scoring mode to BlendedInfixSuggester
        Hide
        ASF subversion and git services added a comment -

        Commit 1721331 from Michael McCandless in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1721331 ]

        LUCENE-6939: add exponential reciprocal scoring mode to BlendedInfixSuggester

        Show
        ASF subversion and git services added a comment - Commit 1721331 from Michael McCandless in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1721331 ] LUCENE-6939 : add exponential reciprocal scoring mode to BlendedInfixSuggester
        Hide
        Michael McCandless added a comment -
        Show
        Michael McCandless added a comment - Thanks Arcadius Ahouansou !
        Hide
        Arcadius Ahouansou added a comment -

        Thank you very much Michael McCandless for your help.
        As part of this change, we would need to update the BlenderType section of the wiki at
        https://cwiki.apache.org/confluence/display/solr/Suggester

        • change linear and reciprocal to position_linear and position_reciprocal respectively, and
        • also add a section for position_exponential_reciprocal and the exponent config

        Again, thank you very much Michael McCandless for your help.

        Show
        Arcadius Ahouansou added a comment - Thank you very much Michael McCandless for your help. As part of this change, we would need to update the BlenderType section of the wiki at https://cwiki.apache.org/confluence/display/solr/Suggester change linear and reciprocal to position_linear and position_reciprocal respectively, and also add a section for position_exponential_reciprocal and the exponent config Again, thank you very much Michael McCandless for your help.
        Hide
        Erick Erickson added a comment -

        Done, but this is all a mystery to me so if I messed it up please let me know.

        Show
        Erick Erickson added a comment - Done, but this is all a mystery to me so if I messed it up please let me know.
        Hide
        Arcadius Ahouansou added a comment -

        Hello Erick Erickson

        I can see you have updated the wiki with linear>position_linear and reciprocal>position_reciprocal

        Thank you very much for that Erick Erickson

        We need a new bullet point for the new blenderType, just below position_reciprocal at the same level:

        position_exponential_reciprocal: weightFieldValue/Math.pow(1+position, exponent). This is a more aggressive and more configurable version of the position_reciprocal with a configurable variable exponent. When exponent==1.0, then both position_exponential_reciprocal and position_reciprocal are equivalent and lead to the same result.

        Then, we will need a new bullet point outside of the BlenderType and just above numFactor:

        exponent: an optional configuration variable for the position_exponential_reciprocal blenderType. It is a decimal number used to control how fast the score will grow or decrease. Its default value 2.0

        Many thanks

        Show
        Arcadius Ahouansou added a comment - Hello Erick Erickson I can see you have updated the wiki with linear > position_linear and reciprocal > position_reciprocal Thank you very much for that Erick Erickson We need a new bullet point for the new blenderType, just below position_reciprocal at the same level: position_exponential_reciprocal : weightFieldValue/Math.pow(1+position, exponent). This is a more aggressive and more configurable version of the position_reciprocal with a configurable variable exponent . When exponent ==1.0, then both position_exponential_reciprocal and position_reciprocal are equivalent and lead to the same result. Then, we will need a new bullet point outside of the BlenderType and just above numFactor: exponent : an optional configuration variable for the position_exponential_reciprocal blenderType. It is a decimal number used to control how fast the score will grow or decrease. Its default value 2.0 Many thanks

          People

          • Assignee:
            Unassigned
            Reporter:
            Arcadius Ahouansou
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development