[STANBOL-1447] Improve FST Corpus handling and updates

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.12.0
• Fix Version/s: 1.0.0, 0.12.1
• Component/s: Enhancement Engines
• Assignee: Rupert Westenthaler
• Reporter: Rupert Westenthaler
• Created: 22/Feb/16 07:43
• Updated: 07/Mar/16 09:46
• Resolved: 07/Mar/16 09:46

Description

The Lucene FST linking engine manages FST corpora that need to be rebuild when the Solr Core has updates. Because of the every FST corpora knows the Solr version it was built for.

Currently it is checked on every call the the Lucene FST linking engine if the version of the FST corpora is still in sync with the version of the the SolrCore. If not the re-creation of the FST corpora is enqueued. However until the conpletion of this task enhancement requests are processed using an older version of the FST corpora.

The initial idea of this was to prevent long waits for huge indexes (e.g. for DBPedia the creation of the FST corpara takes more as a minute). But in reality FST corpora are typically built in seconds.

That means that in most cases it would be better to wait for the re-creation of the corpus rather than using an outdated version.

This issue will change the FST corpora management to use Futures so that components can wait for corpora to be created. The code will also use a reasonable wait time for corpora to be built.