Uploaded image for project: 'Stanbol'
  1. Stanbol
  2. STANBOL-1447

Improve FST Corpus handling and updates

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.12.0
    • Fix Version/s: 1.0.0, 0.12.1
    • Component/s: Enhancement Engines
    • Labels:
      None

      Description

      The Lucene FST linking engine manages FST corpora that need to be rebuild when the Solr Core has updates. Because of the every FST corpora knows the Solr version it was built for.

      Currently it is checked on every call the the Lucene FST linking engine if the version of the FST corpora is still in sync with the version of the the SolrCore. If not the re-creation of the FST corpora is enqueued. However until the conpletion of this task enhancement requests are processed using an older version of the FST corpora.

      The initial idea of this was to prevent long waits for huge indexes (e.g. for DBPedia the creation of the FST corpara takes more as a minute). But in reality FST corpora are typically built in seconds.

      That means that in most cases it would be better to wait for the re-creation of the corpus rather than using an outdated version.

      This issue will change the FST corpora management to use Futures so that components can wait for corpora to be created. The code will also use a reasonable wait time for corpora to be built.

        Attachments

          Activity

            People

            • Assignee:
              rwesten Rupert Westenthaler
              Reporter:
              rwesten Rupert Westenthaler

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment