Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-3475

ShingleFilter should handle positionIncrement of zero, e.g. synonyms

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Patch Available
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: modules/analysis
    • Labels:
    • Lucene Fields:
      New

      Description

      ShingleFilter is creating shingles for a single term that has been expanded by synonyms when it shouldn't. The position increment is 0.

      As an example, I have an Analyzer with a SynonymFilter followed by a ShingleFilter. Assuming car and auto are synonyms, the SynonymFilter produces two tokens and position 1: car, auto. The ShingleFilter is then producing 3 tokens, when there should only be two: car, car auto, auto. This behavior seems incorrect.

        Attachments

        1. LUCENE-3475.patch
          33 kB
          Alan Woodward
        2. LUCENE-3475.patch
          32 kB
          Alan Woodward

          Issue Links

            Activity

              People

              • Assignee:
                romseygeek Alan Woodward
                Reporter:
                cameronl Cameron
              • Votes:
                4 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated: