Lucy
  1. Lucy
  2. LUCY-132

Move Tokenizer, Stemmer, Stopalizer

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.1.0 (incubating)
    • Fix Version/s: 0.1.0 (incubating)
    • Component/s: Analysis
    • Labels:
      None

      Description

      In anticipation of creating interfaces using generic names such as
      "Tokenizer", "Stemmer", and "StopFilter", we should vacate those class names
      and give the current implementations more specific names:

          Lucy::Analysis::Tokenizer  => Lucy::Analysis::RegexTokenizer
          Lucy::Analysis::Stemmer    => Lucy::Analysis::SnowballStemmer
          Lucy::Analysis::Stopalizer => Lucy::Analysis::SnowballStopFilter
      

      Discussion: <http://s.apache.org/RbX>, <http://s.apache.org/WVJ>.

      1. snowball_stemmer.patch
        34 kB
        Marvin Humphrey
      2. snowball_stopfilter.patch
        44 kB
        Marvin Humphrey
      3. regex_tokenizer.patch
        63 kB
        Marvin Humphrey

        Activity

        Marvin Humphrey created issue -
        Hide
        Marvin Humphrey added a comment -

        The attached snowball_stemmer.patch must be applied after running the
        following "svn move" commands:

        svn move core/Lucy/Analysis/Stemmer.c core/Lucy/Analysis/SnowballStemmer.c
        svn move core/Lucy/Analysis/Stemmer.cfh core/Lucy/Analysis/SnowballStemmer.cfh
        svn move core/Lucy/Test/Analysis/TestStemmer.c core/Lucy/Test/Analysis/TestSnowballStemmer.c
        svn move core/Lucy/Test/Analysis/TestStemmer.cfh core/Lucy/Test/Analysis/TestSnowballStemmer.cfh
        svn move perl/lib/Lucy/Analysis/Stemmer.pm perl/lib/Lucy/Analysis/SnowballStemmer.pm
        svn move perl/t/156-stemmer.t perl/t/156-snowball_stemmer.t
        svn move perl/t/core/156-stemmer.t perl/t/core/156-snowball_stemmer.t
        
        Show
        Marvin Humphrey added a comment - The attached snowball_stemmer.patch must be applied after running the following "svn move" commands: svn move core/Lucy/Analysis/Stemmer.c core/Lucy/Analysis/SnowballStemmer.c svn move core/Lucy/Analysis/Stemmer.cfh core/Lucy/Analysis/SnowballStemmer.cfh svn move core/Lucy/Test/Analysis/TestStemmer.c core/Lucy/Test/Analysis/TestSnowballStemmer.c svn move core/Lucy/Test/Analysis/TestStemmer.cfh core/Lucy/Test/Analysis/TestSnowballStemmer.cfh svn move perl/lib/Lucy/Analysis/Stemmer.pm perl/lib/Lucy/Analysis/SnowballStemmer.pm svn move perl/t/156-stemmer.t perl/t/156-snowball_stemmer.t svn move perl/t/core/156-stemmer.t perl/t/core/156-snowball_stemmer.t
        Marvin Humphrey made changes -
        Field Original Value New Value
        Attachment snowball_stemmer.patch [ 12473421 ]
        Hide
        Marvin Humphrey added a comment -

        The attached snowball_stopfilter.patch must be applied after a similar set of
        "svn move" commands.

        Show
        Marvin Humphrey added a comment - The attached snowball_stopfilter.patch must be applied after a similar set of "svn move" commands.
        Marvin Humphrey made changes -
        Attachment snowball_stopfilter.patch [ 12473436 ]
        Hide
        Marvin Humphrey added a comment -

        The attached regex_tokenizer.patch file must be applied after performing a
        similar set of "svn move" commands.

        Show
        Marvin Humphrey added a comment - The attached regex_tokenizer.patch file must be applied after performing a similar set of "svn move" commands.
        Marvin Humphrey made changes -
        Attachment regex_tokenizer.patch [ 12473444 ]
        Hide
        Marvin Humphrey added a comment -

        Committed as r1080735, r1080766, and r1080795.

        Show
        Marvin Humphrey added a comment - Committed as r1080735, r1080766, and r1080795.
        Marvin Humphrey made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Assignee Marvin Humphrey [ creamyg ]
        Resolution Fixed [ 1 ]

          People

          • Assignee:
            Marvin Humphrey
            Reporter:
            Marvin Humphrey
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development