Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-11976

TokenizerChain is overwriting, not chaining TokenFilters in normalize()

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 8.0
    • Fix Version/s: 7.3
    • Component/s: search
    • Labels:
      None

      Description

      TokenizerChain is overwriting, not chaining tokenfilters in normalize.

      This doesn't currently break search because normalize is not being used at the Solr level (AFAICT); rather, TextField has its own analyzeMultiTerm() that duplicates code from the newer normalize.

      Code as is:

          TokenStream result = in;
          for (TokenFilterFactory filter : filters) {
            if (filter instanceof MultiTermAwareComponent) {
              filter = (TokenFilterFactory) ((MultiTermAwareComponent) filter).getMultiTermComponent();
              result = filter.create(in);
            }
          }
      

      The fix is simple:

      -        result = filter.create(in);
      +        result = filter.create(result);
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                dsmiley David Smiley
                Reporter:
                tallison@apache.org Tim Allison
              • Votes:
                1 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 0.5h
                  0.5h