Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-5384

Analysis overview could mention clearAttributes and end

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.7, 6.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      It would be helpful to tokenizer implementors for the analysis package overview to mention more things. I'll supply a patch.

        Activity

        Hide
        bmargulies Benson Margulies added a comment -

        https://github.com/apache/lucene-solr/pull/12 contains some more documentation.

        Yes, this is offered under the terms of the Apache license, in case anyone is still uncertain as to the relationship of github pull requests to the AL.

        Show
        bmargulies Benson Margulies added a comment - https://github.com/apache/lucene-solr/pull/12 contains some more documentation. Yes, this is offered under the terms of the Apache license, in case anyone is still uncertain as to the relationship of github pull requests to the AL.
        Hide
        thetaphi Uwe Schindler added a comment -

        Hi Benson,
        thank you for the patch! I would suggest to change the sentence that mentions clearAttributes() to be more clear, some ideas:

        • every Tokenizer must call clearAttributes (as we have already in the text)
        • every TokenFilter that inserts new tokens into the stream must also call clearAttributes(). Alternatively use captureState()/restoreState() to clone the previous token and modify it afterwards (this is the recommended way to insert tokens from TokenFilters).

        Uwe

        Show
        thetaphi Uwe Schindler added a comment - Hi Benson, thank you for the patch! I would suggest to change the sentence that mentions clearAttributes() to be more clear, some ideas: every Tokenizer must call clearAttributes (as we have already in the text) every TokenFilter that inserts new tokens into the stream must also call clearAttributes(). Alternatively use captureState()/restoreState() to clone the previous token and modify it afterwards (this is the recommended way to insert tokens from TokenFilters). Uwe
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1555907 from Robert Muir in branch 'dev/trunk'
        [ https://svn.apache.org/r1555907 ]

        LUCENE-5384: Add some analysis api tips to the package.html (closes #12)

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1555907 from Robert Muir in branch 'dev/trunk' [ https://svn.apache.org/r1555907 ] LUCENE-5384 : Add some analysis api tips to the package.html (closes #12)
        Hide
        rcmuir Robert Muir added a comment -

        Thanks Benson! Really appreciate the doc contributions.

        Show
        rcmuir Robert Muir added a comment - Thanks Benson! Really appreciate the doc contributions.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1555914 from Robert Muir in branch 'dev/branches/branch_4x'
        [ https://svn.apache.org/r1555914 ]

        LUCENE-5384: Add some analysis api tips to the package.html (closes #12)

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1555914 from Robert Muir in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1555914 ] LUCENE-5384 : Add some analysis api tips to the package.html (closes #12)

          People

          • Assignee:
            bmargulies Benson Margulies
            Reporter:
            bmargulies Benson Margulies
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development