[SOLR-9894] Tokenizer work randomly

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Not A Problem
• Affects Version/s: 6.2.1
• Component/s: query parsers
• Labels: patch

Environment

solrcloud 6.2.1(3 solr nodes)
OS:linux
RAM:8G

• Assignee: Unassigned
• Reporter: 王海涛
• Created: 27/Dec/16 05:39
• Updated: 14/Aug/19 13:18
• Resolved: 10/Aug/19 13:59

Description

my schema.xml has a fieldType as folow:
<fieldType name="my_ik" class="solr.TextField">
<analyzer type="index">
<tokenizer class="org.wltea.analyzer.lucene.IKTokenizerFactory" useSmart="false"/>
<filter class="org.wltea.pinyin.solr5.PinyinTokenFilterFactory" pinyinAll="true" minTermLength="2"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
<analyzer type="query">
<tokenizer class="org.wltea.analyzer.lucene.IKTokenizerFactory" useSmart="true"/>
<filter class="solr.LowerCaseFilterFactory"/>
</analyzer>
</fieldType>

Attention:
index tokenzier useSmart is false
query tokenzier useSmart is true

But when I send query request with parameter q ,
the query tokenziner sometimes useSmart equals true
sometimes useSmart equal false.
That is so terrible!
I guess the problem may be caught by tokenizer cache.
when I query ,the tokenizer should use true as the useSmart's value,
but it had cache the wrong tokenizer result which created by indexWriter who use false as useSmart's value.