Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-1350

Filters which are "consumers" should not reset the payload or flags and should better reuse the token

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: 2.4
    • Labels:
      None
    • Lucene Fields:
      New, Patch Available

      Description

      Passing tokens with payloads through SnowballFilter results in tokens with no payloads.
      A workaround for this is to apply stemming first and only then run whatever logic creates the payload, but this is not always convenient.

      Other "consumer" filters have similar problem.

      These filters can - and should - reuse the token, by implementing next(Token), effectively also fixing the unwanted resetting.

        Attachments

        1. LUCENE-1350.patch
          118 kB
          DM Smith
        2. LUCENE-1350.patch
          0.7 kB
          Doron Cohen
        3. LUCENE-1350-test.patch
          1 kB
          Doron Cohen

          Issue Links

            Activity

              People

              • Assignee:
                doronc Doron Cohen
                Reporter:
                doronc Doron Cohen
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: