Lucene - Core
  1. Lucene - Core
  2. LUCENE-2211

Improve BaseTokenStreamTestCase to uses a fake attribute to check if clearAttributes() was called correctly - found bugs in contrib/analyzers

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.9, 2.9.1, 3.0
    • Fix Version/s: 2.9.2, 3.0.1, 4.0-ALPHA
    • Component/s: modules/analysis
    • Labels:
      None
    • Lucene Fields:
      New, Patch Available

      Description

      Robert had the idea to use a fake attribute inside BaseTokenStreamTestCase that records if its clear() method was called. If this is not the case after incrementToken(), asserTokenStreamContents fails. It also uses the attribute in TeeSinkTokenFilter, because there a lot of copying, captureState and restoreState() is used. By the attribute, you can track wonderful, if save/restore and clearAttributes is correctly implemented. It also verifies that before a captureState() it was also cleared (as the state will also contain the clear call). Because if you consume tokens in a filter, capture the consumed tokens and insert them, the capturedStates must also be cleared before.

      In contrib analyzers are some test that fail to pass this additional assertion. They are not fixed in the attached patch.

      1. LUCENE-2211.patch
        32 kB
        Uwe Schindler
      2. LUCENE-2211.patch
        32 kB
        Robert Muir
      3. LUCENE-2211.patch
        19 kB
        Uwe Schindler
      4. LUCENE-2211.patch
        19 kB
        Uwe Schindler
      5. LUCENE-2211.patch
        18 kB
        Robert Muir
      6. LUCENE-2211.patch
        12 kB
        Robert Muir
      7. LUCENE-2211.patch
        10 kB
        Robert Muir
      8. LUCENE-2211.patch
        8 kB
        Uwe Schindler
      9. LUCENE-2211-branch29.patch
        40 kB
        Uwe Schindler
      10. LUCENE-2211-branch29.patch
        36 kB
        Uwe Schindler
      11. LUCENE-2211-branch30.patch
        34 kB
        Uwe Schindler

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Uwe Schindler
            Reporter:
            Uwe Schindler
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development