[LUCENE-10416] Update Korean Dictionary for Nori

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 10.0 (main)
• Component/s: modules/analysis
• Lucene Fields: New
• Assignee: Unassigned
• Reporter: Uihyun Kim
• Created: 09/Feb/22 15:37
• Updated: 28/Aug/22 16:35
• Resolved: 20/Feb/22 12:52

Description

For Nori - Korean analyzer, there is Korean dictionary named mecab-ko-dic, which is available under an Apache license here: https://bitbucket.org/eunjeon/mecab-ko-dic

 

The dictionary hasn't been updated in Nori although it has some updates to provide better analysis results. Downloading is available here: https://bitbucket.org/eunjeon/mecab-ko-dic/downloads

• Currently used in Nori: https://bitbucket.org/eunjeon/mecab-ko-dic/downloads/mecab-ko-dic-2.0.3-20170922.tar.gz
• Latest: https://bitbucket.org/eunjeon/mecab-ko-dic/downloads/mecab-ko-dic-2.0.3-20170922.tar.gz

 

There are changes between the currently used version and the latest release version(change log: https://bitbucket.org/eunjeon/mecab-ko-dic/src/master/CHANGES.md)

• New feature: added semantic class for NNG - 장소, 행위, 상태변화, 정적상태
• Fix: correct unexpectedly huge cost on NNG/장소
• New words

 

There's no issue with testing :lucene:analysis:nori:test and building a new binary.