Solr
  1. Solr
  2. SOLR-1336

Add support for lucene's SmartChineseAnalyzer

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.1, 4.0-ALPHA
    • Component/s: Schema and Analysis
    • Labels:
      None

      Description

      SmartChineseAnalyzer was contributed to lucene, it indexes simplified chinese text as words.

      if the factories for the tokenizer and word token filter are added to solr it can be used, although there should be a sample config or wiki entry showing how to apply the built-in stopwords list.
      this is because it doesn't contain actual stopwords, but must be used to prevent indexing punctuation...

      note: we did some refactoring/cleanup on this analyzer recently, so it would be much easier to do this after the next lucene update.
      it has also been moved out of -analyzers.jar due to size, and now builds in its own smartcn jar file, so that would need to be added if this feature is desired.

      1. SOLR-1336.patch
        45 kB
        Robert Muir
      2. SOLR-1336.patch
        45 kB
        Robert Muir
      3. SOLR-1336.patch
        4 kB
        Robert Muir

        Activity

        Robert Muir created issue -
        Robert Muir made changes -
        Field Original Value New Value
        Attachment SOLR-1336.patch [ 12415966 ]
        Robert Muir made changes -
        Attachment SOLR-1336.patch [ 12415975 ]
        Robert Muir made changes -
        Attachment SOLR-1336.patch [ 12418535 ]
        Robert Muir made changes -
        Assignee Robert Muir [ rcmuir ]
        Robert Muir made changes -
        Fix Version/s 3.1 [ 12314371 ]
        Fix Version/s 4.0 [ 12314992 ]
        Robert Muir made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Grant Ingersoll made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Robert Muir
            Reporter:
            Robert Muir
          • Votes:
            2 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development