Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-4817

Add KeywordRepeaterFilter to emit tokens twice once as keyword and once not as keyword

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 4.1
    • 4.3, 6.0
    • modules/analysis
    • None
    • New, Patch Available

    Description

      if you want to have a stemmed and an unstemmed version of a token one for recall and one for precision you have to do two fields today in most of the cases. Yet, most of the stemmers respect the keyword attribute so we could add a token filter that emits the same token twice once as keyword and once plain. Folks would most likely need to combine this RemoveDuplicatesTokenFilter but that way we can have stemmed and unstemmed version in the same field.

      Attachments

        1. docs.patch
          5 kB
          Erick Erickson
        2. docs.patch
          5 kB
          Erick Erickson
        3. LUCENE-4817.patch
          10 kB
          Simon Willnauer
        4. LUCENE-4817.patch
          5 kB
          Simon Willnauer

        Issue Links

          Activity

            People

              Unassigned Unassigned
              simonw Simon Willnauer
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Slack

                  Issue deployment