Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-1350

Filters which are "consumers" should not reset the payload or flags and should better reuse the token

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • None
    • 2.4
    • modules/analysis, modules/other
    • None
    • New, Patch Available

    Description

      Passing tokens with payloads through SnowballFilter results in tokens with no payloads.
      A workaround for this is to apply stemming first and only then run whatever logic creates the payload, but this is not always convenient.

      Other "consumer" filters have similar problem.

      These filters can - and should - reuse the token, by implementing next(Token), effectively also fixing the unwanted resetting.

      Attachments

        1. LUCENE-1350.patch
          118 kB
          DM Smith
        2. LUCENE-1350.patch
          0.7 kB
          Doron Cohen
        3. LUCENE-1350-test.patch
          1 kB
          Doron Cohen

        Issue Links

          is part of

          Improvement - An improvement or enhancement to an existing feature or task. LUCENE-1333 Token implementation needs improvements

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. SOLR-532 WordDelimiterFilter ignores payloads

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Activity

            People

              doronc Doron Cohen
              doronc Doron Cohen
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: