[SOLR-1690] JSONKeyValueTokenizerFactory -- JSON Tokenizer

• Type: New Feature
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: Schema and Analysis
• Assignee: Unassigned
• Reporter: Ryan McKinley
• Created: 30/Dec/09 18:42
• Updated: 12/Mar/19 17:29

Description

Sometimes it is nice to group structured data into a single field.

This (rough) patch, takes JSON input and indexes tokens based on the key values pairs in the json.

schema.xml

<!-- JSON Field Type -->
    <fieldtype name="json" class="solr.TextField" positionIncrementGap="100" omitNorms="true">
      <analyzer type="index">
        <tokenizer class="solr.JSONKeyValueTokenizerFactory" keepArray="true" hierarchicalKey="false"/>
        <filter class="solr.TrimFilterFactory"/>
        <filter class="solr.LowerCaseFilterFactory"/>
      </analyzer>
      <analyzer type="query">
        <tokenizer class="solr.KeywordTokenizerFactory"/>
        <filter class="solr.TrimFilterFactory" />
        <filter class="solr.LowerCaseFilterFactory"/>
      </analyzer>
    </fieldtype>

Given text:

 { "hello": "world", "rank":5 }

indexed as two tokens:

term position	1	2
term text	hello:world	rank:5
term type	word	word
source start,end	12,17	27,28