Lucene - Core
  1. Lucene - Core
  2. LUCENE-3755

Add examples to javadocs of Analyzer (4.0)/ReusableAnalyzerBase(3.6)

    Details

    • Type: Task Task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.0, 5.0
    • Component/s: general/javadocs
    • Labels:
    • Lucene Fields:
      New

      Description

      This stuff is great, it makes it easy to define analyzers:

      Analyzer analyzer = new Analyzer() {
          @Override
          protected TokenStreamComponents createComponents(String fieldName, Reader reader) {
            Tokenizer source = new FooTokenizer(reader);
            TokenStream filter = new FooFilter(source);
            filter = new BarFilter(filter);
            return new TokenStreamComponents(source, filter);
          }
        };
      

      But, we should add basic examples to the javadocs I think (we can backport to ReusableAnalyzerBase).
      Also it would be nice to throw in an example that adds a CharFilter.

        Activity

        Robert Muir created issue -
        Robert Muir made changes -
        Field Original Value New Value
        Labels newdev
        Hide
        Steve Rowe added a comment -

        we should add basic examples to the javadocs I think (we can backport to ReusableAnalyzerBase).

        Not sure if it qualifies as "the javadocs" (did you mean the Analyzer class javadocs?), but in LUCENE-3666, with Uwe's help, I updated the o.a.l.analysis package javadocs to include some examples of this, in both trunk and branch_3x.

        Also it would be nice to throw in an example that adds a CharFilter.

        I agree - I came across Analyzer#initReader() after I committed LUCENE-3666, but then forgot to go back and add info about it.

        Show
        Steve Rowe added a comment - we should add basic examples to the javadocs I think (we can backport to ReusableAnalyzerBase). Not sure if it qualifies as "the javadocs" (did you mean the Analyzer class javadocs?), but in LUCENE-3666 , with Uwe's help, I updated the o.a.l.analysis package javadocs to include some examples of this, in both trunk and branch_3x. Also it would be nice to throw in an example that adds a CharFilter. I agree - I came across Analyzer#initReader() after I committed LUCENE-3666 , but then forgot to go back and add info about it.
        Hide
        Robert Muir added a comment -

        OK, LUCENE-3666 did look to do some nice cleanup (sorry i think the package-level javadocs are often forgotten).

        I did notice this in the javadocs there:

        Analyzer myAnalyzer = new StandardAnalyzer() {
        

        No way that can work

        But, any silly nitpicking aside, I think the package-level javadocs are very nice at introducing the analysis package.
        Yet we could imagine there are two levels of expertise here:

        1. making your own analyzer from lucene's existing components (there are so many after all)
        2. understanding the entire analysis api enough to implement your own components

        So I think a good balance could be to add a simple basic example to Analyzer.java/ReusableAnalyzerBase.java's class javadocs,
        but refer/link? to the package-level javadocs for additional use-cases like using charfilters or implementing your own filters.

        Show
        Robert Muir added a comment - OK, LUCENE-3666 did look to do some nice cleanup (sorry i think the package-level javadocs are often forgotten). I did notice this in the javadocs there: Analyzer myAnalyzer = new StandardAnalyzer() { No way that can work But, any silly nitpicking aside, I think the package-level javadocs are very nice at introducing the analysis package. Yet we could imagine there are two levels of expertise here: making your own analyzer from lucene's existing components (there are so many after all) understanding the entire analysis api enough to implement your own components So I think a good balance could be to add a simple basic example to Analyzer.java/ReusableAnalyzerBase.java's class javadocs, but refer/link? to the package-level javadocs for additional use-cases like using charfilters or implementing your own filters.
        Hide
        Uwe Schindler added a comment -

        +1

        I also need to change package javadocs about the Reader refactoring... I still see IndexReader.open() in official Javadocs...

        Show
        Uwe Schindler added a comment - +1 I also need to change package javadocs about the Reader refactoring... I still see IndexReader.open() in official Javadocs...
        Hide
        Steve Rowe added a comment -

        I did notice this in the javadocs there: Analyzer myAnalyzer = new StandardAnalyzer() { No way that can work

        Right, I missed that one.

        So I think a good balance could be to add a simple basic example to Analyzer.java/ReusableAnalyzerBase.java's class javadocs, but refer/link? to the package-level javadocs for additional use-cases like using charfilters or implementing your own filters.

        +1

        Show
        Steve Rowe added a comment - I did notice this in the javadocs there: Analyzer myAnalyzer = new StandardAnalyzer() { No way that can work Right, I missed that one. So I think a good balance could be to add a simple basic example to Analyzer.java/ReusableAnalyzerBase.java's class javadocs, but refer/link? to the package-level javadocs for additional use-cases like using charfilters or implementing your own filters. +1
        Robert Muir made changes -
        Fix Version/s 3.6 [ 12319070 ]
        Robert Muir made changes -
        Fix Version/s 4.1 [ 12321140 ]
        Fix Version/s 4.0 [ 12314025 ]
        rmuir committed 1376614 (1 file)
        Reviews: none

        LUCENE-3755: add simple example to javadocs of Analyzer and ref package docs for more examples

        rmuir committed 1376615 (4 files)
        Robert Muir made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Fix Version/s 5.0 [ 12321663 ]
        Fix Version/s 4.0 [ 12322550 ]
        Fix Version/s 4.1 [ 12321140 ]
        Resolution Fixed [ 1 ]
        Hide
        Uwe Schindler added a comment -

        Closed after release.

        Show
        Uwe Schindler added a comment - Closed after release.
        Uwe Schindler made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Unassigned
            Reporter:
            Robert Muir
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development