[STANBOL-593] EntityIterator implementation based on Jena TDB that allows to filter Entities based on Triple Filters

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.9.0-incubating
• Fix Version/s: entityhub-0.10.0-incubating
• Component/s: Entityhub
• Assignee: Rupert Westenthaler
• Reporter: Rupert Westenthaler
• Created: 24/Apr/12 18:47
• Updated: 10/Jul/12 10:57
• Resolved: 27/Apr/12 06:22

Description

The FieldValueProcessor (EntityProcessor) already allows to filter Entities based on Triple Filters. However this requires to Iterate over all entities - something very ineffective if one wants only to index a rather small fraction of all Entities.

To achieve better performance in such cases one needs an Component that uses a similar functionality to filter Entities within the Indexing Source. Such an implementation is very easy to implement based on Jena TDB as the low level API natively supports filtered iterators.

Indexing configurations would than use a EntityIterator/EntityDataProvider combination as source for the indexing. A according configuration would look like

entityIdIterator=org.apache.stanbol.entityhub.indexing.source.jenatdb.ResourceFilterIterator,config:entityTypes.properties
entityDataProvider=org.apache.stanbol.entityhub.indexing.source.jenatdb.RdfIndexingSource,source:rdfdata

the entityTypes.properties file would require the following properties

field=rdf:type
values=dbp-ont:Person;dbp-ont:Place;dbp-ont:Organisation

With this configuration the indexing process would only iterate over Persons, Places and Organisations present within the IndexingSource.