Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.0-ALPHA
    • Component/s: modules/analysis
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      Many TokenStreams created in tests are not reusable. Some do some really messy things which prevent their reuse so we may have to change the tests themselves.

      We'll target back porting this to 3x.

      1. LUCENE-3397.patch
        29 kB
        Chris Male
      2. LUCENE-3397.patch
        29 kB
        Chris Male
      3. LUCENE-3397-more.patch
        6 kB
        Chris Male
      4. LUCENE-3397-highlighter.patch
        0.9 kB
        Chris Male

        Activity

        Hide
        Chris Male added a comment -

        All TokenStreams are now reusable.

        Show
        Chris Male added a comment - All TokenStreams are now reusable.
        Hide
        Simon Willnauer added a comment -

        thanks for all this cleanup work chris! this makes our codebase much cleaner and test real examples! Very very welcome work!

        Show
        Simon Willnauer added a comment - thanks for all this cleanup work chris! this makes our codebase much cleaner and test real examples! Very very welcome work!
        Hide
        Chris Male added a comment -

        Committed revision 1161993.

        Show
        Chris Male added a comment - Committed revision 1161993.
        Hide
        Chris Male added a comment -

        Patch addresses the SynonymTokenizer (note, not a Tokenizer) in HighlighterTest which was only resetting its actual Tokenizer once.

        Show
        Chris Male added a comment - Patch addresses the SynonymTokenizer (note, not a Tokenizer) in HighlighterTest which was only resetting its actual Tokenizer once.
        Hide
        Chris Male added a comment -

        Found another. Okay I'm going to stop resolving this till I'm sure we're clear.

        Show
        Chris Male added a comment - Found another. Okay I'm going to stop resolving this till I'm sure we're clear.
        Hide
        Chris Male added a comment -

        Committed revision 1161986.

        I understand Robert is backporting the changes.

        If I find anymore, I'll address them in LUCENE-3396.

        Show
        Chris Male added a comment - Committed revision 1161986. I understand Robert is backporting the changes. If I find anymore, I'll address them in LUCENE-3396 .
        Hide
        Chris Male added a comment -

        Patch which addresses the remaining non-reusable TSs.

        Patch also takes a swing at the deprecated PatternAnalyzer's Tokenizers (even though they were neither Tokenizers, nor in tests).

        Show
        Chris Male added a comment - Patch which addresses the remaining non-reusable TSs. Patch also takes a swing at the deprecated PatternAnalyzer's Tokenizers (even though they were neither Tokenizers, nor in tests).
        Hide
        Chris Male added a comment -

        Found some more.

        Show
        Chris Male added a comment - Found some more.
        Hide
        Chris Male added a comment -

        Committed revision 1161488.

        I or someone else will get to back porting this at another stage.

        Show
        Chris Male added a comment - Committed revision 1161488. I or someone else will get to back porting this at another stage.
        Hide
        Robert Muir added a comment -

        looks good!

        Show
        Robert Muir added a comment - looks good!
        Hide
        Chris Male added a comment -

        Better patch which removes changing the state of a static variable.

        I'll commit this in day or so.

        Show
        Chris Male added a comment - Better patch which removes changing the state of a static variable. I'll commit this in day or so.
        Hide
        Chris Male added a comment -

        Patch which adds state resetting to TokenStreams created in tests.

        The only other TSs which are not reusable are done so by design to test that something can handle non-reusable TSs. When we made reuse mandatory, we can drop these tests and remove the non reusable TSs.

        Show
        Chris Male added a comment - Patch which adds state resetting to TokenStreams created in tests. The only other TSs which are not reusable are done so by design to test that something can handle non-reusable TSs. When we made reuse mandatory, we can drop these tests and remove the non reusable TSs.

          People

          • Assignee:
            Chris Male
            Reporter:
            Chris Male
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development