Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-4479

TokenSources.getTokenStream() doesn't return correctly for termvectors with positions but no offsets

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 4.0
    • Fix Version/s: 4.1, 6.0
    • Component/s: modules/highlighter
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      The javadocs for TokenSources.getTokenStream(Terms, boolean) state:

      "Low level api. Returns a token stream or null if no offset info available
      in index. This can be used to feed the highlighter with a pre-parsed token
      stream"

      However, if the Terms instance passed in has positions but no offsets stored, a TokenStream is incorrectly returned, rather than null.

      This has the effect of incorrectly highlighting fields with term vectors and positions, but no offsets. All highlighting markup is prepended to the beginning of the field.

        Attachments

        1. LUCENE-4479.patch
          13 kB
          Alan Woodward
        2. LUCENE-4479.patch
          12 kB
          Alan Woodward
        3. LUCENE-4479.patch
          12 kB
          Alan Woodward
        4. LUCENE-4479.patch
          3 kB
          Alan Woodward

          Activity

            People

            • Assignee:
              romseygeek Alan Woodward
              Reporter:
              romseygeek Alan Woodward
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: