[SOLR-3792] NullPointerException in highlighter under certain conditions

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Information Provided
• Affects Version/s: 4.0-BETA
• Component/s: highlighter

Environment

Windows 7, Java 6 or 7

• Assignee: Unassigned
• Reporter: Yoni Amir
• Created: 05/Sep/12 14:08
• Updated: 14/Oct/16 11:12
• Resolved: 14/Oct/16 11:12

Description

Under certain conditions, there is an exception in highlighter component. The stacktrace is shown below.
The conditions are:
1) Add a document with a field that has empty value. E.g., with solrj api:

document.addField ("field1", "");

(Maybe this is not a valid value or use-case, but Solr stills allows it!!)

2) have solr.WhitespaceTokenizerFactory as the tokenizer of the analyzer chain for that field.

3) make sure that this field should be highlighted. In my case, I was using hl.requireFieldMatch=false, and actually I was searching on another field.

4) Using edismax, search for a phrase, e.g., "foo bar" (including the quotation marks)

5) The document mentioned before should be part of the search results.

6) This exception occurs:

INFO (SolrCore.java:1670) - [rcmCore] webapp=/solr path=/select params=

{qf=all_text&version=2&rows=20&wt=javabin&start=0&q="foo bar"}

hits=103 status=500 QTime=38 ERROR (SolrException.java:104) - null:java.lang.NullPointerException
at org.apache.lucene.analysis.util.CharacterUtils$Java5CharacterUtils.fill(CharacterUtils.java:191)
at org.apache.lucene.analysis.util.CharTokenizer.incrementToken(CharTokenizer.java:152)
at org.apache.lucene.analysis.miscellaneous.WordDelimiterFilter.incrementToken(WordDelimiterFilter.java:209)
at org.apache.lucene.analysis.util.FilteringTokenFilter.incrementToken(FilteringTokenFilter.java:50)
at org.apache.lucene.analysis.miscellaneous.RemoveDuplicatesTokenFilter.incrementToken(RemoveDuplicatesTokenFilter.java:54)
at org.apache.lucene.analysis.core.LowerCaseFilter.incrementToken(LowerCaseFilter.java:54)
at org.apache.solr.highlight.TokenOrderingFilter.incrementToken(DefaultSolrHighlighter.java:629)
at org.apache.lucene.analysis.CachingTokenFilter.fillCache(CachingTokenFilter.java:78)
at org.apache.lucene.analysis.CachingTokenFilter.incrementToken(CachingTokenFilter.java:50)
at org.apache.lucene.search.highlight.Highlighter.getBestTextFragments(Highlighter.java:225)
at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlightingByHighlighter(DefaultSolrHighlighter.java:510)
at org.apache.solr.highlight.DefaultSolrHighlighter.doHighlighting(DefaultSolrHighlighter.java:401)
at org.apache.solr.handler.component.HighlightComponent.process(HighlightComponent.java:136)
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:206)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1656)
at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:454)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:275)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)
at java.lang.Thread.run(Thread.java:736)