[STANBOL-980] Add Japanese Language support by using the Solr/Lucene Kuromoji Analyzer

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: commons-0.11.0
• Fix Version/s: 0.12.0
• Component/s: Commons, Enhancer
• Assignee: Rupert Westenthaler
• Reporter: Rupert Westenthaler
• Created: 11/Mar/13 10:45
• Updated: 23/Jul/13 14:03
• Resolved: 20/Mar/13 14:45

Description

With the most recent Solr/Lucene versions the Kuromoji Analyzer for Japanese was added. This module will allow to

• index and search Entities with Japanese language labels and texts
• Tokenize Japanese Text
• POS tagging of Japanese Text
• NER for Persons, Organizations and Places
• Lemmatization
• Correct Label Tokenization required for linking Japanese labels of Entities

This will required three modules:

• extension to the commons.solr.core module that provide the Kuromoji Analyzer as Bundle
• NLP processing Engine
• LabelTokenizer implementation

In addition an own bundlelist that includes those three modules. This Bundlelist should be added by default to the Full Stanbol Launcher.