Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-3475

ShingleFilter should handle positionIncrement of zero, e.g. synonyms

Add voteWatch issue
    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Patch Available
    • Minor
    • Resolution: Unresolved
    • None
    • None
    • modules/analysis
    • New

    Description

      ShingleFilter is creating shingles for a single term that has been expanded by synonyms when it shouldn't. The position increment is 0.

      As an example, I have an Analyzer with a SynonymFilter followed by a ShingleFilter. Assuming car and auto are synonyms, the SynonymFilter produces two tokens and position 1: car, auto. The ShingleFilter is then producing 3 tokens, when there should only be two: car, car auto, auto. This behavior seems incorrect.

      Attachments

        1. LUCENE-3475.patch
          33 kB
          Alan Woodward
        2. LUCENE-3475.patch
          32 kB
          Alan Woodward

        Issue Links

          is part of

          Bug - A problem which impairs or prevents the functions of the product. LUCENE-4170 TestRandomChains fail with Shingle+CommonGrams

          • Major - Major loss of function.
          • Open

          Activity

            People

              romseygeek Alan Woodward
              cameronl Cameron
              Votes:
              4 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:

                Slack

                  Issue deployment