Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-1350

Filters which are "consumers" should not reset the payload or flags and should better reuse the token

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Duplicate
    • None
    • 2.4
    • None
    • New, Patch Available

    Description

      Passing tokens with payloads through SnowballFilter results in tokens with no payloads.
      A workaround for this is to apply stemming first and only then run whatever logic creates the payload, but this is not always convenient.

      Other "consumer" filters have similar problem.

      These filters can - and should - reuse the token, by implementing next(Token), effectively also fixing the unwanted resetting.

      Attachments

        1. LUCENE-1350-test.patch
          1 kB
          Doron Cohen
        2. LUCENE-1350.patch
          0.7 kB
          Doron Cohen
        3. LUCENE-1350.patch
          118 kB
          DM Smith

        Issue Links

          Activity

            People

              doronc Doron Cohen
              doronc Doron Cohen
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: