Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-2211

Improve BaseTokenStreamTestCase to uses a fake attribute to check if clearAttributes() was called correctly - found bugs in contrib/analyzers

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.9, 2.9.1, 3.0
    • 2.9.2, 3.0.1, 4.0-ALPHA
    • modules/analysis
    • None
    • New, Patch Available

    Description

      Robert had the idea to use a fake attribute inside BaseTokenStreamTestCase that records if its clear() method was called. If this is not the case after incrementToken(), asserTokenStreamContents fails. It also uses the attribute in TeeSinkTokenFilter, because there a lot of copying, captureState and restoreState() is used. By the attribute, you can track wonderful, if save/restore and clearAttributes is correctly implemented. It also verifies that before a captureState() it was also cleared (as the state will also contain the clear call). Because if you consume tokens in a filter, capture the consumed tokens and insert them, the capturedStates must also be cleared before.

      In contrib analyzers are some test that fail to pass this additional assertion. They are not fixed in the attached patch.

      Attachments

        1. LUCENE-2211-branch29.patch
          40 kB
          Uwe Schindler
        2. LUCENE-2211-branch29.patch
          36 kB
          Uwe Schindler
        3. LUCENE-2211-branch30.patch
          34 kB
          Uwe Schindler
        4. LUCENE-2211.patch
          32 kB
          Uwe Schindler
        5. LUCENE-2211.patch
          32 kB
          Robert Muir
        6. LUCENE-2211.patch
          19 kB
          Uwe Schindler
        7. LUCENE-2211.patch
          19 kB
          Uwe Schindler
        8. LUCENE-2211.patch
          18 kB
          Robert Muir
        9. LUCENE-2211.patch
          12 kB
          Robert Muir
        10. LUCENE-2211.patch
          10 kB
          Robert Muir
        11. LUCENE-2211.patch
          8 kB
          Uwe Schindler

        Activity

          People

            uschindler Uwe Schindler
            uschindler Uwe Schindler
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: