[SOLR-3724] No highlighting for phrases with stop words when FVH is used

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 3.6.1
• Component/s: highlighter
• Assignee: Unassigned
• Reporter: Igor Motov
• Created: 09/Aug/12 14:29
• Updated: 04/Jan/18 22:00
• Resolved: 04/Jan/18 22:00

Description

To reproduce:

• Index text "foo and bar" into the field "message" with the following schema :

  <schema name="example" version="1.5">
    <types>
      <!-- ... -->
      <fieldType name="my_text_general" class="solr.TextField" positionIncrementGap="100">
        <analyzer type="index">
          <tokenizer class="solr.StandardTokenizerFactory"/>
          <filter class="solr.StopFilterFactory" ignoreCase="true" words="lang/stopwords_en.txt" enablePositionIncrements="true"/>
          <filter class="solr.LowerCaseFilterFactory"/>
        </analyzer>
        <analyzer type="query">
          <tokenizer class="solr.StandardTokenizerFactory"/>
          <filter class="solr.StopFilterFactory" ignoreCase="true" words="lang/stopwords_en.txt" enablePositionIncrements="true"/>
          <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" ignoreCase="true" expand="true"/>
          <filter class="solr.LowerCaseFilterFactory"/>
        </analyzer>
      </fieldType>
      <!-- ... -->
    </types>
    <fields>
      <!-- ... -->
      <field name="message" type="my_text_general" indexed="true" stored="true" required="true" termVectors="true" termPositions="true" termOffsets="true"/>
      <!-- ... -->
    </fields>
    <!-- ... -->
  </schema>
  

• Search for the message:"foo and bar" with highlighting enabled and hl.useFastVectorHighlighter=true
• The text is not highlighted

Standard highlighter works fine. If I set enablePositionIncrements=false in the analyzer, FVH starts to highlight the entire phrase. You can find complete schema and test data files that I used to reproduce this issue here: https://gist.github.com/3279879

Issue Links

• duplicates: LUCENE-7551 FastVectorHighlighter ignores position in PhraseQuery