Lucene - Core
  1. Lucene - Core
  2. LUCENE-3091

Highlighter's TokenSources should use TermVector positions if indexed

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: modules/highlighter
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      Spinoff from LUCENE-3087

      TokenSources is able to use positions indexed into TermVectors, if you pass "false" for the tokenPositionsGuaranteedContiguous argument to getTokenStream.

      But if you pass true, it doesn't use the positions. Maybe, we are not allowed (supposed) to pass true if the posIncr could be 0? But, if that is OK/allowed, then we should fix the code to handle the posIncr=0 case correctly.

      LUCENE-3087 made one simple fix, to guess at the posInc (1 or 0) based on the offsets of adjacent tokens.

        Activity

        Michael McCandless created issue -

          People

          • Assignee:
            Unassigned
            Reporter:
            Michael McCandless
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:

              Development