[STANBOL-1125] Create a lightweight EntityHub Indexing Tool for Freebase

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Won't Fix
• Component/s: Entityhub
• Assignee: Rafa Haro
• Reporter: Rafa Haro
• Created: 25/Jun/13 11:20
• Updated: 03/Jul/15 23:37
• Resolved: 03/Jul/15 23:37

Description

Due to the enormous size of the dumps, current Freebase indexing tool in Stanbol can't barely work in machines without several gigas of RAM and/or SSD disks. JenaTDB importer has been identified as the bootle neck of the indexing process. To use an RDF database is mandatory in order to, for instance, use LDPath programs at indexing time.

The idea is to develop a lightweight indexing tool that stream data from the dumps and push it directly to Solr. Despite losing some functionality, it is possible for any user to generate Freebase EntityHub indexes from any dump.

Issue Links

• relates to: STANBOL-1014 Create Entityhub Indexing Tool for freebase.com