Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-8719

FixedShingleFilter can miss paths at the end of a TokenStream

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 8.1
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      If we have a set of paths of differing lengths at the end of a TokenStream, FixedShingleFilter will currently stop when the first path hits the end of the stream, but this may miss extra longer paths.  For example, the stream "a b:3/c d e f" with a shingle size of 3 should produce paths "a b f", "a c d", "c d e" and "d e f", but at the moment FixedShingleFilter will stop when it finds that the path "b f" is too short to produce a shingle.

        Attachments

        1. LUCENE-8719.patch
          6 kB
          Alan Woodward

          Activity

            People

            • Assignee:
              romseygeek Alan Woodward
              Reporter:
              romseygeek Alan Woodward
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: