Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-6468

Regression: StopFilterFactory doesn't work properly without deprecated enablePositionIncrements="false"

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 4.8.1, 4.9, 5.3.1, 6.6.2, 7.1
    • Fix Version/s: None
    • Component/s: Schema and Analysis
    • Labels:
      None

      Description

      Setup:

      • Schema version is 1.5
      • Field config:
        <fieldType name="words_ngram" class="solr.TextField" omitNorms="false" autoGeneratePhraseQueries="true">
          <analyzer>
            <tokenizer class="solr.PatternTokenizerFactory" pattern="[^\w]+" />
            <filter class="solr.StopFilterFactory" words="url_stopwords.txt" ignoreCase="true" />
            <filter class="solr.LowerCaseFilterFactory" />
          </analyzer>
        </fieldType>
        
      • Stop words:
        http 
        https 
        ftp 
        www
        

      So very simple. In the index I have:

      • twitter.com/testuser

      All these queries do match:

      • twitter.com/testuser
      • com/testuser
      • testuser

      But none of these does:

      Debug output shows:
      "parsedquery_toString": "+(url_words_ngram:\"? twitter com testuser\")"
      But we need:
      "parsedquery_toString": "+(url_words_ngram:\"twitter com testuser\")"

      Complete debug outputs:

      The complete discussion and explanation of the problem is here: http://lucene.472066.n3.nabble.com/Help-with-StopFilterFactory-td4153839.html

      I didn't find a clear explanation how can we upgrade Solr, there's no any replacement or a workarround to this, so this is not just a major change but a major disrespect to all existing Solr users who are using this feature.

        Attachments

        1. FieldValue.png
          15 kB
          Elvis Rocha

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                aheaven Alexander S.
              • Votes:
                3 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated: