Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-4479

TokenSources.getTokenStream() doesn't return correctly for termvectors with positions but no offsets

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 4.0
    • 4.1, 6.0
    • modules/highlighter
    • None
    • New

    Description

      The javadocs for TokenSources.getTokenStream(Terms, boolean) state:

      "Low level api. Returns a token stream or null if no offset info available
      in index. This can be used to feed the highlighter with a pre-parsed token
      stream"

      However, if the Terms instance passed in has positions but no offsets stored, a TokenStream is incorrectly returned, rather than null.

      This has the effect of incorrectly highlighting fields with term vectors and positions, but no offsets. All highlighting markup is prepended to the beginning of the field.

      Attachments

        1. LUCENE-4479.patch
          3 kB
          Alan Woodward
        2. LUCENE-4479.patch
          12 kB
          Alan Woodward
        3. LUCENE-4479.patch
          12 kB
          Alan Woodward
        4. LUCENE-4479.patch
          13 kB
          Alan Woodward

        Activity

          People

            romseygeek Alan Woodward
            romseygeek Alan Woodward
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment