[SOLR-1535] Pre-analyzed field type

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.5
• Fix Version/s: 4.0-ALPHA
• Assignee: Andrzej Bialecki
• Reporter: Andrzej Bialecki
• Created: 30/Oct/09 17:52
• Updated: 20/Jun/14 16:01
• Resolved: 19/Apr/12 15:10

Description

PreAnalyzedFieldType provides a functionality to index (and optionally store) content that was already processed and split into tokens using some external processing chain. This implementation defines a serialization format for sending tokens with any currently supported Attributes (eg. type, posIncr, payload, ...). This data is de-serialized into a regular TokenStream that is returned in Field.tokenStreamValue() and thus added to the index as index terms, and optionally a stored part that is returned in Field.stringValue() and is then added as a stored value of the field.

This field type is useful for integrating Solr with existing text-processing pipelines, such as third-party NLP systems.

Issue Links

• is related to: SOLR-4532 Log pre-analyzed documents
• relates to: SOLR-4619 Improve PreAnalyzedField query analysis
• relates to: SOLR-5915 Cannot set parserImpl=... with PreAnalyzedField