Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-5278

MockTokenizer throws away the character right after a token even if it is a valid start to a new token

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Trivial
    • Resolution: Fixed
    • None
    • 4.6, 6.0
    • None
    • None
    • New

    Description

      MockTokenizer throws away the character right after a token even if it is a valid start to a new token. You won't see this unless you build a tokenizer that can recognize every character like with new RegExp(".") or RegExp("...").

      Changing this behaviour seems to break a number of tests.

      Attachments

        1. LUCENE-5278.patch
          5 kB
          Nik Everett
        2. LUCENE-5278.patch
          6 kB
          Robert Muir
        3. LUCENE-5278.patch
          9 kB
          Robert Muir

        Activity

          People

            rcmuir Robert Muir
            nik9000@gmail.com Nik Everett
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: