Lucene - Core
  1. Lucene - Core
  2. LUCENE-3106

commongrams filter calls incrementToken() after it returns false

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.2, 4.0-ALPHA
    • Component/s: modules/analysis
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      In LUCENE-3064, we beefed up MockTokenizer with assertions, and I started cutting over some analysis tests to use MockTokenizer for better coverage.

      The commongrams tests fail, because they call incrementToken() after it already returns false.

      In general its my understanding consumers should not do this (and i know of a few tokenizers that will actually throw exceptions if you do this, just like java iterators and such).

      1. LUCENE-3106_test.patch
        5 kB
        Robert Muir
      2. LUCENE-3106.patch
        6 kB
        Robert Muir

        Issue Links

          Activity

          Hide
          Robert Muir added a comment -

          patch with the test modifications to produce the failure.

          Show
          Robert Muir added a comment - patch with the test modifications to produce the failure.
          Hide
          Robert Muir added a comment -

          here's the obvious solution, but there might be a cleaner way to rewrite its loop...

          Show
          Robert Muir added a comment - here's the obvious solution, but there might be a cleaner way to rewrite its loop...
          Hide
          Robert Muir added a comment -

          bulk move 3.2 -> 3.3

          Show
          Robert Muir added a comment - bulk move 3.2 -> 3.3
          Hide
          Robert Muir added a comment -

          this was fixed in LUCENE-3113

          Show
          Robert Muir added a comment - this was fixed in LUCENE-3113
          Hide
          Uwe Schindler added a comment -

          close issue, as fixed in already released version

          Show
          Uwe Schindler added a comment - close issue, as fixed in already released version

            People

            • Assignee:
              Unassigned
              Reporter:
              Robert Muir
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development