[STANBOL-1128] Implement a Lucene FST based Entity Linking Engine (based on OpenSextant / SolrTextTagger)

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Component/s: Enhancement Engines
• Assignee: Rupert Westenthaler
• Reporter: Rupert Westenthaler
• Created: 27/Jun/13 10:19
• Updated: 09/Oct/13 09:28
• Resolved: 03/Oct/13 06:39

Description

This will implement an in-memory EntityLinking EnhancementEngine based on Lucenes FST (Finite State Transducer) technology.

This engine could make direct use of the Classes contained in the OpenSextant / SolrTextTagger code [1]. This uses a two layered FST

(1) to represent words and
(2) to map those words to phrases

With this it is possible to efficiently hold big vocabularies in memory (> 300MByte for geonames.org). See the presentation at [2] for more details.

While the license is fully compatible (ASL 2.0) the library is currently not available on Maven Central. We need to contact the author regarding this.

[1] https://github.com/OpenSextant/SolrTextTagger
[2] http://www.lucenerevolution.org/2013/Text-Tagging-with-Finite-State-Transducers

Issue Links

• relates to: STANBOL-1153 Improve Solr schema used by the Entityhub SolrYard
• relates to: STANBOL-1167 SolrYard IndexingDestination should support creation of FST models
• requires: STANBOL-1148 Update the dbpedia default data